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FINAL  REPORT 
PREFACE 


This  document  is  the  Final  Report,  CDRL  A008,  produced  as 
part  of  the  Interactive  Computer  Program  Development  System 
Study  for  the  Defense  Mapping  Agency.  An  Executive  Summary 
is  provided  at  the  beginning  of  the  report  to  provide  a 
concise  description  of  the  major  aspects  of  the  study.  The 
tools  and  equipment  recommended  as  a  result  of  this  study  are 
the  ones  which  best  satisfied  the  requirements  and 
constraints  of  the  Defense  Mapping  Agency  (DMA)  environment 
at  the  time  this  document  was  produced. 
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EXECUTIVE  SUMMARY 
for  the 

DEFENSE  MAPPING  AGENCY 
MODERN  PROGRAMMING  ENVIRONMENT 
SPECIFICATION 


I.  Abstract 

This  summary  provides  a  synopsis  of  the  technical 
requirements  specification,  a  cost  estimate,  and  an 
implementation  schedule  plan  for  a  Modern  Programming 
Environment  (MPE)  for  the  Defense  Mapping  Agency  (DMA).  The 
conclusions  and  recommendations  stated  in  this  summary  are 
the  results  of  the  Interactive  Computer  Program  Development 
System  Study  performed  by  General  Dynamics  Data  Systems 
Division  (3D/DSD)  under  contract  F30602-81-C-0039  to  Rome  Air 
Development  Center  (RADC) .  The  objectives  of  this  study  were: 

1.  To  identify  DMA  needs  for  a  Modern 
Programming  Environment.’ 

2.  To  formulate  a  total  systems  concept  to 
satisfy  the  identified  needs. 

3.  To  survey  and  evaluate  software  tool 
candidates  for  the  Modern  Programming 
Environment.  . 

4.  To  specify  a  Modern  Programming  Environment  and 

an  implementation  plan  that  satisfies  DMA  needs.  C — 

The  study  was  conducted  with  full  cognizance  of  both  recent 
in-house  DMA  activities  such  as  the  Software  Improvement 
Program  (SIP) ,  and  currently  contracted  system  development 
efforts;  for  example,  the  Digital  Stereo  Comparator  Compiler, 
TES/EMPS,  Universal  Rectifier,  and  the  Clustered  Carto 
System.  The  study  conclusions  and  recommendations  are 
compatible  with  these  in-house  and  contracted  efforts. 

The  primary  contract  deliverables  are  three  reports:  (1)  a 
Functional  Description  of  the  MPE,  (2)  a  System/Subsystem 
Specification  that  details  the  MPE  configuration  and 
identifies  particular  software  tools,  and  (3)  a  Final  Report 
that  summarizes  all  stages  of  the  study  and  provides  cost  and 
schedule  estimates. 

The  recommended  MPE  configuration  is  a  network  of  VAX-11/780 
computers  that  support  ANSI  FORTRAN  and  COBOL  software 
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lifecycle  tool  environments  for  software  development  and 
maintenance.  This  VAX  network  has  telecommunication  links 
with  production  mainframe  and  individual  minicomputer 
targets.  At  the  time  of  the  preparation  of  this  report,  the 
VAX-11/780  computer  is  the  state-of-the-art  technology  that 
best  satisfies  the  requirements  of  the  DMA  MPE.  Therefore, 
the  VAX-11/780  computer  will  be  referenced  as  the  tool 
bearing  host  throughout  this  report.  The  MPE  configuration 
can  easily  support  the  Ada*  language  should  the  Defense 
Mapping  Agency  employ  Ada  in  future  work.  It  is  recommended 
that  a  full  MPE  configuration  be  duplicated  at  both  DMAHTC 
and  DMAAC  to  enhance  the  utilization  of  common  software 
between  centers. 

It  is  estimated  that  the  total  cost  (hardware  ■  i  software 
procurements  plus  contractor  development)  for  im  'mentation 
of  the  DMA  MPE  is  $11  million.  These  funds  expended 

during  a  54  month  implementation  period  that  star  in  fiscal 
year  1983  and  ends  in  fiscal  year  1988.  The  im  -''ntation 
plan  consists  of  four  phases: 

1.  Phase  I  -  Near-term  experimental  system 

2.  Phase  IA  -  Near-term  full-scale  system 

3.  Phase  II  -  Far-term  experimental  system 

4.  Phase  I I A  -  Far-term  full-scale  system 


This  implementation  plan  includes  DMA  decision  points  for 
continuation  of  work  authorization  and  assures  a  working 
system  is  available  at  the  end  of  each  phase.  The  outlook  is 
for  experimental  near-term  capabilities  to  be  available  in 
1985  and  then  evolving  to  full  far-term  capabilities  in  1987. 

The  benefits  of  the  Modern  Programming  Environment  to  the 
Defense  Mapping  Agency  are  twof old--cost  and  technical 
capability.  First,  it  is  estimated  that  the  entire  $11 
million  implementation  cost  is  recovered  within  five  years 
(in  fiscal  year  1988)  from  the  start  of  implementation  and  in 
five  more  years  (in  fiscal  year  1993)  the  cumulative  net 
savings  of  the  Modern  Programming  Environment  is  $25  million. 
Secondly,  the  Modern  Programming  Environment  provides  the 
tools,  methodologies,  and  guidelines  to  meet  the  increasing 
strategic  and  tactical  requirements  for  the  processing  of 
digital  data  which  would  be  impossible  to  meet  using  existing 
methods. 

*  Ada  is  a  registered  trademark  of  the  U.S.  Government  (AJPO) 


II.  Technical  Summary  of  the  Modern  programming  Environment 

There  are  several  components  to  a  modern  programming 
environment.  The  relationships  among  these  components  can  be 
represented  by  a  "layered  model"  wherein  the  set  of  all 
interior  layers  supports  the  next  outermost  layer.  The 
particular  components  and  layering  for  the  recommended 
Defense  Mapping  Agency  Modern  Programming  Environment  are 
shown  in  Figure  I.  It  is  recommended  that  both  DMAHTC  and 
DMA AC  have  this  MPE  configuration  to  facilitate  the  use  of 
common  software. 


Figure  I:  DEFENSE  MAPPING  AGENCY  MODERN  PROGRAMMING 
ENVIRONMENT  CONFIGURATION 


The  core  of  the  DMA  MPE  is  a  network  of  VAX-11/780  computers 
that  serves  as  the  tool  bearing  host  for  the  FORTRAN  and 
COBOL  software  tool  environments.  The  Ada  environment  is 
being  developed  under  tri-servic^s  sponsorship,  and  it  should 
be  available  to  the  DMA  MPE  as  a  government-owned  environment 
if  DMA  uses  Ada  in  future  work.  This  situation  is 
represented  by  the  dotted  line  around  the  Ada  portion  of 
Figure  I.  The  principal  factors  that  led  to  the  selection  of 
the  VAX-11/780  as  the  tool  bearing  host  are: 

1.  A  full  complement  of  lifecycle  tools  that 
supports  DMA's  needs  for  software  develop¬ 
ment  and  maintenance  already  exists. 
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Vendor  support  for  the  VAX  system  is  excellent. 

Much  government,  military,  and  commercial 
software  RED  efforts  are  already  targeted  to  the 
VAX  and  more  such  efforts  are  expected  in  the 
future.  Examples  of  organizations  that  already 
have  VAX  based  system  development  environments 
include:  Bell  Research  Labs,  TRW,  Air  Force 
Wright  Aeronautical  Laboratory,  General  Research 
Corporation  and  Boeing.  Therefore,  DMA  can 
upqrade  their  MPE  tool  set  in  the  future  at 
little  or  no  additional  cost. 

3.  The  cost  and  facilities  requirements  for  a  VAX  system 
are  considerably  less  than  a  mainframe  computer  tool 
bearing  host. 

4.  The  DMA  already  has  several  contracted  efforts, 
namely,  TES/EMPS,  Clustered  Car*o,  and  PAHS 

that  are  based  on  a  VAX-11/780  system.  Hence,  the 
maintenance  of  these  contractor  developed  systems 
by  DMA  using  a  compatible  VAX  based  MPE  will  be  very 
cost  effective. 

Table  I  shows  the  recommended  set  of  tools  for  the  DMA  MPE 
that  supports  the  complete  software  lifecycle;  that  is,  the 
requirements,  desiqn,  coding,  testing,  and  maintenance  phases 
as  well  as  the  project  management  and  training  activities. 
These  tools  constitute  the  second  layer  in  Figure  I. 


SOFTWARE  TOOL  1 

BUFFO  ATS  IFC  CYCLE 

PHASE  FUNCTIONS 

•OSI.IT 
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•FORTRAN  77 
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UmmtHm  Appfcaaaaaa 

CaAiap 

•COBOL  M 

-  ANSI  SmMail  Hrpfcv  0 rim  liapaap  la 

Iwmw  ApplwaUaaa 

•A* 

-  0 apart ■  ami  •«  Oafaaaa  laaAil  H«pAar  OrAar 
tampmapa 

CaAiap 

•FAVt/RXVMt 

T—» 

•CAWS 
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I»»l 
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•APSE 
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•VUI 
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SckaAaAap  SptM 

•HYPE  RSRAFHICS  -  Mtmmmmmm  |bm A  Syoma  far  fnpmttM 
mi  Mnnwoh  mf  Tamtwi  mi  6«ffaa  HmtW 
OmAmo  M  VAX  ■  tar-tana) 

Tmaiap 

TABLE  I:  DEFENSE  MAPPING  AGENCY  MODERN  PROGRAMMING  ENVIRONMENT 
SOFTWARE  TOOLS 
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Bach  of  these  tools  wt  .3  d  be  hosted  on  the  VAX  in  the  far- 
term;  consequently,  the  DMA  MPE  has  the  benefit  of  being  a 
largely  stand-alone  software  development  and  maintenance 
facility. 

To  describe  "how  to"  effectively  use  the  capabilities  of  the 
DMA  MPE,  the  particular  sequence  and  conditions  in  which  the 
individual  software  tools  would  be  used  has  been  aodeled. 
This  model  included  five  software  development  and  maintenance 
scenarios: 


Scenario  1  -  Maintenance  of  existing  software  which 

has  not  been  Software  Improvement  Program 
(SIP)  upgraded. 

Scenario  2  -  Maintenance  of  existing  software 
which  has  been  SIP  upgraded. 

Scenario  3  -  Software  presently  under  development 

for  which  standards  were  not  specified. 

Scenario  U  -  New  software  to  be  developed  by  DMA  for 
which  standards  will  be  specified. 

Scenario  5  -  New  software  to  be  developed  by  contractors 
for  which  standards  will  be  specified. 


An  overview  of  the  descriptions  of  the  software  tool  use  that 
is  common  to  all  of  these  scenarios  is  shown  in  Figure  II. 


SCENARIO 

TOOLS 

SOFTWARE 

DETERMINATION 

APPROACH 

TESTING 

Figure  II:  OVERVIEW  OF  SOFTWARE  TOOL  USE  FOR  DEFENSE  MAPPING  AGENCY 
MODERN  PROGRAMMING  ENVIRONMENT 

Two  important  features  of  the  detailed  scenario  descriptions 
are:  (1)  all  software  lifecycle  phases  are  supported  by 
automated  tools  and  (2)  there  are  only  two  tool  apt>roaches--a 
conventional  tools  approach  and  an  automatic  programming 
approach.  These  scenarios  will  form  the  basis  for  the 
toolsmith  and  personnel  training  layer  of  Figure  I. 

Detailed  methodologies,  software  development  standards  and 
guidelines,  and  training  course  development  will  be  part  of 
the  planned  implementation  follow-on  task  and  are  not 
contained  in  this  document.  However,  the  draft  versions  of 
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the  DMA  Software  Life  Cycle  standards  prepared  under  the 
auspices  of  the  SIP  effort  are  entirely  compatible  with  this 
MPE  specification.  It  is  expected  that  continued  co¬ 
ordination  between  SIP  and  MPE  will  result  in  a  set  of 
Software  Life  Cycle  Standards  that  is  supported  by  the  MPE 
capabilities  and  conversely. 

The  principal  technical  benefits  of  the  recommended  DMA  MPE 
are : 

1.  The  stand-alone  characteristic  of  the  VAX 
network  plus  software  tool  complement  will 
permit  easy  training  of  personnel  and  high 
programmer  productivity  in  software  development 
and  maintenance  efforts. 

2.  State-of-the-art  software  tools  are 
available  now  for  the  VAX  and  the  trend 
is  to  continue  tool  developments  for 
VAX  systems. 

3.  The  network  capability  permits  easy 
growth  as  DMA  processing  requirements 
increase. 

4.  The  VAX  based  MPE  is  inherently  compatible 
with  several  new  systems  that  are  now  under 
development.  Software  maintenance  of  these 
systems  using  the  MPE  will  be  facilitated. 


Ill 


•  Schedule_and_Cost_Esti mates  for  the  Modern 
na_En  vironmen  t_Im£lemen  ta  t  ion 

The  recomnendation  for  the  inplementation  of  the  DMA  Modern 
Programming  Environment  as  specified  in  this  document  is  a 
four-phased  program  spanning  54  calendar  months  (July  1983  to 
December  1987) .  The  schedules  and  tasks  for  each  phase  are 
shown  in  Figure  III. 


MONTHS 

DMA  OECISION  PTS 
•PHASE  I 
•PHASE  IA 
•PHASE  II 
•PHASE  IIA 
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21  24  27  30 
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MONTHS 
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•USE.IT,  SDDL,  VUE 

-  Evalnatiaa.  Trainini,  ft  Mathodalo|ias 

-  Oaupn  Naar-Tvm  Fuli  Scala  Syitam 

-  Preliminary  Dasifn  Far-Tarm 
Exparimantal  Syitam 
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15  18  21  24  27 

A 

A _ t 

\ 

1  A _ 

i 

L 

i 
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PHASE 

TASKS 

1 

•OESIGN.  IMPLEMENTATION, ft  TRAINING  FOR  NEAR-TERM  EXPERIMENTAL. SYSTEM 
•  OESIGN  NEAR-TERM  FULL-SCALE  SYSTEM 
•DEVELOP  METHODOLOGIES 

•PRELIMINARY  DESIGN  OF  FAR-TERM  EXPERIMENTAL  SYSTEM 

IA 

•IMPLEMENTATION  OF  NEAR-TERM  FULL-SCALE  SYSTEM 
•NETWORK  VAXi  AND  LINK  TO  MAINFRAME 
•TRAINING  ON  NEAR-TERM  FULL-SCALE  SYSTEM 
•UPGRADE  METHODOLOGIES 

II 

•DESIGN  AND  IMPLEMENT  FAR-TERM  EXPERIMENTAL  SYSTEM 
•DESIGN  FAR-TERM  FULL-SCALE  SYSTEM 
•INTEGRATE  SOFTWARE  TOOLS 
•UPGRADE  METHODOLOGIES 
•IDENTIFY  R&D  EFFORTS 

IIA 

•IMPLEMENT  FAR-TERM  FULL-SCALE  SYSTEM 

•  TRAINING  ON  FAR-TERM  FULL-SCALE  SYSTEM 

•  FINALIZE  METHODOLOGIES 

Figure  III:  TASKS  AND  SCHEDULES  FOR  DEFENSE  MAPPING  AGENCY  MODERN 
PROGRAMMING  ENVIRONMENT  IMPLEMENTATION 
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This  four-phased  schedule  plan  has  several  benefits  to  DMA, 
in  particular: 


1.  There  is  low  risk  because  of  the  careful 
interleaving  of  implementation  and  design 
tasks  within  each  phase  as  shown  in  Table  II. 
An  implementation  task  is  always  preceded  by 
its  corresponding  design  task. 


PHASE 

IMPLEMENTATION  TASK 

0ESI6N  TASK 

1 

NEAR  TERM  EXPERIMENTAL  SYSTEM 

NEAR-TERM  FULL-SCALE  SYSTEM 
FAR-TERM  EXPERIMENTAL  SYSTEM 

IA 

NEAR  TERM  FULL-SCALE  SYSTEM 

II 

FAR  TERM  EXPERIMENTAL  SYSTEM 

FAR-TERM  FULL-SCALE  SYSTEM 

IIA 

FAR  TERM  FULL-SCALE  SYSTEM 

TABLE  II:  INTERLEAVING  OF  SYSTEM  IMPLEMENTATION  AND  DESIGN  TASKS 
FOR  THE  DEFENSE  MAPPING  AGENCY  MODERN  PROGRAMMING 
ENVIRONMENT 


2.  Each  phase  ends  with  a  viable  product.  At  the 
end  of  each  phase  the  DMA  has  a  working  system 
for  their  continued  evaluation. 

3.  Decision  points  are  included  at  which  time 
DMA  can  evaluate  the  progress  to 

date,  provide  direction,  and  authorize  continua¬ 
tion  of  work. 

4.  Phase  I  has  been  carefully  planned  to  provide  the 
maximum  benefits  from  the  resources  required.  The 
principal  benefits  of  Phase  I  are:  (a)  Only  one  VAX 
system  and  software  tool  set  will  be  procured,  yet 
tool  support  will  be  available  for  all  software 
lifecycle  phases  within  15  calendar  months  of  con¬ 
tract  start;  (b)  a  link  with  the  production  mainframe 
computer  will  be  accomplished  to  early  establish  MPE 
compatibility  with  the  mainframe;  (c)  a  total  of  21 
calendar  months  will  be  available  for  training, 
methodology  development  and  evaluation,  and  (d)  designs 
for  Phase  IA  and  Phase  II  will  be  completed  to  provide 
DMA  early  insight  to  the  full  MPE  implementation. 

The  total  cost  for  the  implementation  of  the  DMA  MPE  is 
estimated  to  be  $11  million.  This  estimate  includes  the 
procurement  of  9  VAX-11/780  computers  and  associated 

terminals  and  hardware,  7  software  tool  sets,  maintenance  for 
the  hardwaie  and  software  procurements,  and  contractor  labor. 


The  hardware  procurements  cost  approximately  $4.2  million; 
software  procurements  cost  approximately  $2.5  million,  and 
contractor  labor  costs  approximately  $4.3  million.  The  cost 
estimates  by  phase,  funding  type,  and  fiscal  year  are  shown 
in  Table  III. 


FUNDING 

COSTS  (in  Thousands  of  Dollars) 

TOTALS 

TYPE 

FY  83 

FY84 

FY  85 

FY  86 

FY  87 

FY  88 

BY  PHASE 

1 

Naw-Ttrrn  Experimental  System 

R&O 

BO 

1,210 

090 

1,990 

IA 

Near-Twin  Full-Sal*  System 

Prod  uc  tie* 

1,490 

3,030 

120 

5,240 

II 

Far-Twin  Experimental  System 

R&D 

770 

730 

1,500 

IIA 

Fm-Twm  Full-Sal*  System 

Production 

1.870 

5B0 

2,250 

TOTALS  BY  FUNDING  TYPE 

R&D 

Production 

90 

1,210 

600 

1,490 

770 

3,630 

730 

1.790 

500 

3.490 

7.490 

TOTAL  BY  FISCAL  YEAR 

TOTAL 

90 

1,210 

2.180 

4.400 

2,520 

580 

10.980 

TABLE  III:  COST  ESTIMATES  FOR  IMPLEMENTATION  OF  DEFENSE  MAPPING 
AGENCY  MODERN  PROGRAMMING  ENVIRONMENT 


An  estimate  of  the  savings  realized  by  using  the  MPE  as 
compared  to  continued  use  of  existing  DMA  methods  of  software 
development  and  maintenance  was  also  calculated.  The  inputs 
to  the  savings  estimate  include  productivity  improvements  due 
to  the  software  tools,  percentage  of  DMA  activity  in  each 
lifecycle  phase,  the  DMA  programming  population,  an  estimate 
for  percentage  savings  as  a  function  of  time,  and  the  DMA 
workyear  cost  including  inflation.  The  cumulative  net 
savings  (cumulative  net  savings  =  sum  of  (yearly  savings  - 
yearly  costs) )  due  to  the  DMA  HPE  capabilities  is  shown  in 


Figur*  IV:  CUMULATIVE  NET  SAVINGS  REALIZED  WITH  THE  DEFENSE 
MAPPING  AGENCY  MODERN  PROGRAMMING  ENVIRONMENT 
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The  costs  of  the  DMA  HPE  will  be  recovered  after  five  years 
(in  1988),  and  after  ten  years  (in  1993)  an  estimated 
cuaulative  net  savings  of  $25  Billion  vill  be  realized.  The 
DIM  Modern  Prograaaing  Environment  is  definitely  cost 
effective. 
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IV.  guamrY.of  ..Ggnqral  Dynamics'  Technical  Approach 


General  Dynaaics  Data  Systeas  Division  (GD/DSD)  accoaplished 
the  Defense  Happing  Agency  Hodern  Prograaaing  Environaent 
study  in  21  aonths  (froa  January  1981  to  Septeaber  1982) 
using  a  four  stage  approach.  The  study  ailestone  schedule 
and  a  block  diagraa  of  the  stages  of  the  technical  approach 
are  shown  in  Figure  V. 


1981 

1982 

J  FMAMJ  JASOND 

J  F  M  A  M  J  J  A  ]  S 

STAGE  1 

STAGE  2 

STAGE  Z 

STAGE  4 

A  -ti  >FR 

60  Tool  Pmnt  DMA  Tool 

E*tl 

Figure  V:  GENERAL  DYNAMICS'  TECHNICAL  APPROACH  TO  THE  DEFENSE  MAPPING 
AGENCY  MODERN  PROGRAMMING  ENVIRONMENT  STUDY 

The  objective  of  the  first  stage  of  the  study  was  to  identify 
the  Defense  Mapping  Agency  needs  for  a  aodern  prograaaing 
environaent.  General  Dynamics  distributed  a  five-part 
questionnaire  to  aanagement  and  technical  personnel  at  DMAAC, 
DHAHTC,  and  DMAHQ  to  ascertain  the  basic  data  for 
identification  of  DMA  needs.  A  total  of  181  questionnaires 
were  returned.  Personal  interviews  with  DMA  representatives 
were  then  conducted  to  gain  additional  insights  into  DMA 
needs.  Stage  1  concluded  with  a  list  of  40  generic  needs. 
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categorized  by  software  lifecycle  phase,  and  weighted  by 
importance. 

In  Stage  2  of  the  study,  23  system  operational  concepts  were 
formulated  that  satisfied  the  identified  needs.  A  matrix  of 
needs  vs.  concepts  was  created  to  specify  which  needs  were 
satisfied  by  which  concepts. 

In  Stage  3,  25  different  software  tools  were  presented  and 
demonstrated  at  DMAAC  and  DMAHTC  during  a  two  week  period  at 
each  center.  The  DMA  comments  from  these  presentations  were 
analyzed,  and  8  tools  that  supported  all  software  lifecycle 
phases  were  selected  for  an  in-depth,  8  week  DMA  evaluation 
at  each  center.  The  test-bed  used  for  the  evaluation  process 
was  the  digital  land  mass  problem.  Once  again,  the  DMA 
comments  were  collected  and  analy?ed.  Subsequently,  the 
USE. IT  software  tool  was  of  particular  interest  to  the  MPE 
study  participants  because  of  its  requirements  definition, 
design,  and  automated  FORTRAN  coding  capabilities.  Hence,  it 
was  decided  to  evaluate  the  applicability  of  USE. IT  to  the 
DMA  software  environment  by  solving  a  realistic  DMA  problem. 
The  chosen  problem  was  a  long  range  navigation  (LOFAN) 
lattice  calculation,  and  the  evaluation  was  conducted  from 
July  to  September,  1982.  During  this  period  the  LOFAN 
problem  was  modeled  with  USE. IT,  executable  code  was 
produced,  and  the  graphics  displays  were  demonstrated  to  DMA. 
Stage  3  ended  with  a  "best-case"  modern  programming 
environment  model.  This  best-case  was  formulated  by  rating 
all  available  tools  that  satisified  the  needs  and  concepts 
identified  in  Stages  1  and  2  and  then  selecting  those  tools 
that  best  satisfied  these  needs  and  concepts.  The  rating  was 
accomplished  by  using  the  concept  implementation  evaluation 
sheets  to  ensure  traceability  to  needs  and  concepts,  proper 
weighting  of  evaluation  criteria,  and  consistency.  A  total 
of  173  concept  implementation  evaluation  sheets  were 
completed. 

Finally,  in  Stage  4  this  best-case  modern  program  environment 
model  was  modified  to  satisfy  the  objectives  and  constraints 
of  the  near-term  and  far-term  DMA  modern  programming 
environment.  Typical  constraints  included  cost,  maturity  of 
tools,  availability  of  tools  on  the  tool  bearing  host, 
continued  support  of  DMA  FORTRAN  and  COBOL  efforts,  user- 
friendliness  of  tools,  vendor  support,  logical  integration  of 
tools  to  support  the  entire  lifecycle  phase,  and  smooth 
transition  from  the  near-term  to  the  far-term  configuration. 
An  additional  consideration  was  the  impact  of  the  DMA 
Software  Improvement  Program  (SIP)  upon  the  specification  of 
the  modern  programming  environment.  The  objectives  of  the 
SIP  were  identified  and  found  to  support  the  system 
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operational  concepts  of  the  MPE.  Therefore,  compatibility  of 
the  MPE  with  SIP  was  a  goal  in  the  formulation  of  the  near- 
term  and  far-term  DMA  modern  programming  environment 
configurations.  Stage  4  culminated  in  the  specification  of 
the  Defense  Mapping  Agency  Modern  Programming  Environment 
configuration  as  shown  in  Figure  I  and  the  software  tool  list 
as  shown  in  Table  I. 

A  vitally  important  constituent  of  the  General  Dynamics' 
technical  approach  was  the  continual  interaction  among 
General  Dynamics,  DMA,  and  RADC.  General  Dynamics  spent  92 
workdays  (during  63  calendar  days)  on-site  at  DHAHTC  and 
DMAAC  for  technical  interchange  and  data  gathering.  An 
additional  14  separate  trips  were  made  to  DMAHTC,  DMAAC, 
DMAHQ,  and  RADC  for  status  reviews,  oral  presentations,  and 
documentation  preparation.  Telephone  communications  among 
General  Dynamics,  DMA,  and  RADC  were  extensively  used  to  keep 
all  team  members  abreast  of  the  Modern  Programming 
Environment  project  status.  These  activities  ensure  that  our 
DMA  Modern  Programming  Environment  specification  will  satisfy 
DMA  needs  and  will  be  compatible  with  future  DMA  plans. 
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V .  conclusions  and  Recommendations 

The  fundamental  conclusion  of  this  Interactive  Computer 
Program  Development  System  Study  is  that  the  Modern 
Programming  Environment  specified  in  this  document  satisfies 
known  Defense  Mapping  Agency  requirements  for  the 
introduction  of  state-of-the-art  software  engineering 
technology  into  DMA's  operational  procedures.  In  addition, 
the  plan  for  the  implementation  of  this  Modern  Programming 
Environment  is  orderly,  well-structured,  and  cost  effective. 

The  major  benefits  to  the  DMA  of  this  Modern  Programming 
Environment  specification  are: 

1.  The  technical  hardware/software  configuration 
is  flexible  and  can  easily  grow  and  adapt  to 
future  DMA  needs. 

2.  The  VAX  based  MPE  provides  the  technology  base 
for  rapid  realization  of  productivity 
improvement  in  both  software  development  and 
maintenance. 

3.  The  MPE  will  pay  for  itself  in  five  years  and 
continue  to  accumulate  net  savings  every  year 
thereafter. 

4.  The  introduction  of  the  MPE  will  not  disrupt 
DMA’s  production  operations. 

A  modern  programming  environment  is  more  than  software  tools 
hosted  on  a  computer  system.  The  tool  bearing  host  computer 
and  the  complement  of  software  tools  shown  in  Figure  I  and 
Table  I  form  the  core  of  the  MPE.  It  is  recommended  that  the 
following  items  be  considered  as  part  of  the  total  scope  of 
the  DMA  Modern  Programming  Environment: 

1.  Management  directives  and  commitments  to 
the  development  and  support  of  the  MPE 
are  required  to  ensure  continuity 
of  the  MPE  across  all  DMA  software 
development  and  maintenance  efforts.  In 
particular,  software  contractors 
need  DMA  management  direction  to  use 
development  techniques  and  tools  that 
enable  easy  DMA  maintenance  of  the 
delivered  software  using  the  MPE. 


2. 


The  establishment  of  standards,  guidelines 
reviews,  and  methodologies  for  software 


development  and  maintenance  ace  needed. 

The  work  begun  in  these  areas  by  the 
Software  Improvement  Program  is  the  correct 
first  step.  There  needs  to  be  a  continua¬ 
tion  of  the  already  existing  co-ordination 
between  the  Software  Improvement  Program 
and  the  Modern  Programming  Environment 
study/implementation . 

3.  Personnel  training  in  the  proper  and  efficient 
use  of  the  software  tools  is  vital  to  realize 
the  estimated  productivity  improvements. 

Training  is  a  short-term  cost  with  many-fold, 
long-term  benefits. 

Finally,  as  the  result  of  our  study.  General  Dynamics 
recommends  the  Defense  Mapping  Agency  proceed  with  the 
implementation  of  the  Modern  Programming  Environment 
specified  in  this  document. 
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1.0  INTRODUCTION 


General  Dynamics  Data  Systems  Division  (GD/DSD)  was 
contracted  to  perform  an  Interactive  Computer  Program 

Development  System  Study  (ICPDSS)  for  the  Defense  Happing 
ency  (DMA)  under  contract  to  Rome  Air  Development  Center 
(RADC) .  As  a  result  of  this  study  GD/DSD  has  developed  a 
complete  design  specification  for  a  modern  programming 
environment  (HPE)  for  use  by  DMA  by  1987.  The  technical 

approach  to  the  study  was  organized  into  four  distinct 

stages: 

1)  Determination  of  Defense  Mapping  Agency  needs 

2)  Formulation  of  system  concepts  to  satisfy  those  needs 

3)  Creation  of  the  best-case  model  for  a  modern 
programming  environment 

4)  Application  of  constraints  to  this  model  to  arrive  at 
near-term  (1985)  and  far-term  (1987)  system 
recommendations. 

This  process  is  illustrated  in  Figure  1.1.  In  the 
implementation  of  this  process  close  co-ordination  and 
communication  was  planned  and  established  with  each  of  the 
two  DMA  centers. 

Sections  1  and  2  of  the  report  describe  Stage  one  of  the 
technical  approach,  and  the  development  of  the  DMA  Statement 
of  Operational  Need  (SON)  (Figure  1.2).  The  sources  of 
information  for  the  SON  were  government  documents  dealing 
with  previous  DMA  studies,  a  GD/DSD  survey  guestionnaire ,  and 
personnel  interviews  conducted  by  GD/DSD  project  team  members 
at  the  DMA  Hydrographic/Topographic  Center  (DMAHTC)  and  the 
DMA  Aerospace  Center  (DMAAC) .  The  results  of  the 
questionnaire  were  evaluated  using  a  database  inquiry  system. 
These  results  along  with  an  additional  list  of  needs  derived 
from  government  documents  were  used  in  formulating  the 
original  SON.  The  SON  lists  DMA  needs  and  rates  them  on  a  1 
to  5  scale  where  1  implies  a  low  need  and  5  a  high  need. 
The  columns  present  the  ratings  as  determined  by  each  center 
and  by  General  Dynamics.  The  process  used  is  described  in 
detail  in  Section  2.0  of  this  report.  In  June  1981,  meetings 
were  held  at  each  center  to  validate  the  original  SON 
findings.  These  meetings  resulted  in  revisions  to  the  SON 
and  are  described  fully  in  Section  3.0. 

Stage  two,  described  in  Sections  4  through  7,  involved  the 
development  of  a  SON/SOC  matrix  providing  a  mapping  of  the 
operation  needs  identified  in  the  SON  into  one  or  more 
generic  programming  concepts  which  satisfy  each  need.  A 
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complete  explanation  of  the  development  of  this  matrix  and 
how  to  read  and  use  it  is  found  in  Section  4.0.  On  18-19 
August  1981  the  SON/SOC  matrix  was  presented  at  an  In- 
Process-Review  (IPR)  at  RADC.  As  a  result  of  this  review 
several  changes  were  made  to  the  SON  and  the  SOC's.  The 
changes,  described  in  Section  5.0,  have  been  incorporated  and 
the  current  SON  and  SON/SOC  matrix  are  what  appears  in 
Figures  1.2  and  1.3. 


25 


Figure  1.1  Technical  App 
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Basic  BBTXIG 


CATEGORf 

PROJECT  BANAGEBENT 

DHAAC 

DSD 

DBA  BTC 

10  IHPROVED  BILESTONE  IDENTIFICATION 

3 

3 

5 

12  IBPROVE  BANLOADING 

0 

0 

0 

14  IHPBOVE  SCHEDULE  IBPACT  ANALYSIS 

3 

It 

5 

08  CHARGEBACK  SYSTEfl 

3 

5 

3 

56  BANAGEBENT  TRACKING  FUNCTIONS 

REOUI REBENTS 

3 

5 

3 

1  FORBAL  REQUIRES  ENTS  SPECIFICATION 

0 

3 

5 

5  REQUIREBENTS  TRACKING 

3 

3 

4 

57  SOFTWARE  DEVELOPHENT  TOOLS 

5 

5 

5 

59  STANDARDIZED  PHASED  DEVELOPHENT 

DESIGN 

3 

5 

3 

21  SIBULATOR  FOR  DESIGN 

3 

3 

4 

22  PROGRAB  DESIGN  LANGUAGE 

0 

0 

5 

57  SOFTWARE  DEVELOPHENT  TOOLS 

5 

5 

5 

59  STANDARDIZED  PHASED  DEVELOPHEIT 

CODING 

3 

5 

3 

55  BODERR  SOURCE  DATA  ENTRY  TECHNIQUES 

5 

5 

5 

57  SOFTWARE  DEVELOPHENT  TOOLS 

5 

5 

5 

59  STANDARDIZED  PBASED  DEVELOPHENT 

TEST 

3 

5 

3 

2  QA  PROCEDURES  AND  GUIDELINES 

3 

5 

5 

21  SIBULATOR  FOR  DESIGN 

3 

3 

0 

36  GRAPHICS  AIDS 

5 

0 

5 

57  SOFTWARE  DEVELOPHENT  TOOLS 

5 

5 

5 

59  STANDARDIZED  PHASED  DEVELOPHENT 

3 

5 

3 

BAI NTENANCE  _ 

9  COR  FIGURATION  CONTROL  5 

•0  HISTORICAL  DATA  BASB  TECHNIQUES  3 

57  SOFTWARE  DEVELOPHENT  TOOLS  5 

58  PRODUCTION  PROGRAB  OPTIHIZITION  3 

59  ST ANDABDIZBD  PHASED  DEVELOPHENT  3 

OTHER _ 

3  INTERACTIVE  STSTEH  ACCESS  5 

0  INCREASED  ROBBER  OF  TERBIRALS  5 

11  DECREASED  PAPERSORX  3 

16  UPDATE  OF  OLD  DOCOBERTATXOI  5 

. 18  FASTER  INTEGRATION  OF  REN  BHPLOTEES  3 

34  AUTOHATED  TEXT  BANAGEHRNT  TOOL  3 

01  ORGANIZATION  TOOLS/TZCHHIQDES  IRTEIFACB  3 
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A  very  important  step  in  Stage  3  of  this  Interactive  Computer 
Program  Development  System  Study  was  the  evaluation  and 
selection  of  software  tools  for  the  DMA  MPE.  The  software 
tools  of  interest  in  this  study  provide  the  capabilities  and 
functions  as  outlined  in  the  following  paragraphs. 

Automated  software  development  tools  serve  as  aids  in  the 
support  of  the  software  life  cycle  requirements,  design, 
programming,  testing  and  maintenance  phases.  These  tools  are 
provided  to  assist  the  manager,  designer  and  programmer  by 
automating  part  of  the  development  process.  Automation  not 
only  increases  productivity,  but  it  improves  reliability  and 
quality  by  using  sound,  well  tested  procedures  with  each 
program  developed.  Automated  software  tools  provide  an 
effective  way  to  implement  standards  and  conventions,  and  it 
improves  the  opportunity  to  reuse  software  and  to  reduce 
development  costs.  A  general  overview  discussion  of 
automated  software  tools  is  provided  for  information  and 
insight  into  what  is  currently  available  and  what  is  possible 
for  future  extensions. 

Requirements  tools  allow  a  user  to  document  and,  in  some 
implementations,  analyze  requirements  in  a  succinct  and 
unambiguous  form.  When  analysis  is  possible  a  data  base  is 
constructed  which  is  examined  for  consistency,  completeness 
and  traceability.  USE. IT  is  such  a  tool.  A  requirements 
specification  data  base  is  built  using  a  prompted, 
interactive  interface  language  called  AXES.  Part  of  the 
output  of  USE. IT  is  a  set  of  documents  containing  graphic  and 
textual  descriptions  of  the  requirements  of  a  specified 
software  system  in  a  format  consistent  with  any  other 
software  system  modeled  using  the  tool.  This  documentation 
can  then  be  used  as  input  into  the  design  phase  of  the 
software  life  cycle. 

Design  tools  allow  the  user  to  document  a  design  and  perform 
an  analysis  to  determine  if  it  is  technically  viable.  Both 
processes  are  only  partially  automated  except  in  extremely 
narrow  applications.  SDDL  is  a  design  tool  which  performs 
these  functions.  A  language  based  system,  SDDL,  documents 
the  design  in  a  concise  structured  syntax  which  is  used  to 
perform  a  small,  but  high  level  analysis.  The  information 
provided  by  the  analysis,  however,  greatly  decreases  the 
effort  required  to  manually  evaluate  the  design's  technical 
merit.  A  manual  conversion  of  data  would  be  required  to 
convert  the  USE. IT  output  into  a  format  acceptable  to  SDDL. 
Once  complete,  the  design  is  manually  translated  into  a 
computer  program  by  use  of  a  specified  language.  If  the 
target  language  is  known  prior  to  the  design  phase  (e.g.. 
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FORTRAN  or  COBOL)  SDDL  can  be  utilized  in  a  Banner  that 
decreases  the  effort  needed  in  the  translation  process. 

Once  translated  into  a  computer  program  a  software  system 
must  then  be  processed  by  language  packages  which  are  tools 
that  transform  the  system  into  a  state  understandable  by  a 
digital  computer.  Compilers  represent  a  major  tool  category 
within  the  realm  of  language  packages.  A  compiler  ties  a 
specific  implementation  of  a  high  level  language  to  a 
specific  computer  architecture  (i.e.,  the  operations  executed 
by  the  hardware) . 

Testing  tools  are  used  to  evaluate  the  quality  of  software 
and  demonstrate  that  it  fulfills  the  needs  documented  in  the 
user  specifications.  Testing  tool  capabilities  include,  but 
are  not  limited  to,  static  analysis  (performed  on  code 
including  checks  for  program  structure,  complexity,  and 
format) ,  dynamic  analysis  (performed  during  program  execution 
includes  coverage  analysis  and  assertion  checking) ,  automated 
test  data  generation,  and  output  comparators. 

Maintenance  is  the  life  cycle  phase  when  software  is  placed 
in  operational  use.  Tools  used  during  this  phase  assist 
programmers  in  repairing  or  modifying  existing  production 
software  systems.  Repair  and  modification  are  primarily 
redevelopment  activities  which  can  be  accommodated  for  the 
most  part  by  the  planned  reuse  of  most  or  alJ  of  those 
development  tools  used  during  the  requirements,  design, 
coding,  and  testing  phases.  Additional  maintenance  tools 
include  configuration  control  tools  that  are  used  to  control 
changes  to  a  production  system  and  its  documentation  once  it 
has  been  baselined.  Configuration  control  tools  maintain  the 
current  status  proqram  and  it's  documentation  along  with  the 
past  history  of  all  code  and  documentation  generated  and 
changed. 

The  previously  described  software  life  cycle  presupposes  a 
defined  problem  exists  which  is  known  to  be  solvable.  A 
corollary  of  this  fact  is  that  a  problem  definition  step 
actually  exists.  Problem  definition  always  occurs  prior  to 
requirement  specification,  however,  analysis  of  solvability 
rarely  occurs.  One  reason  behind  this  fact  is  the  labor 
intensive  characteristic  of  feasibility  studies.  USE. IT  also 
automates  this  process.  Additional  output  of  this  tool  not 
previously  mentioned  allows  the  specification  of  a  system  in 
a  very  high  level  language  which  is  automatically  translated 
into  an  executable  form.  In  this  manner  a  rapid  prototype  of 
a  system  may  be  generated  and  studied  for  technical 
viability. 
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As  mentioned  previously,  part  of  the  work  to  be  accomplished 
in  the  ICPDSS  was  the  evaluation  and  selection  of  software 
development  tools  to  be  included  in  the  DBA  near-term  and 
far-term  BPB’s.  This  tool  selection  survey  was  divided  into 
two  phases.  The  goal  of  Phase  1  was  the  selection, 
demonstration  and  evaluation  of  a  large  number  of  software 
tools  that  are  applicable  to  the  DBA  environment.  The  goal 
of  Phase  II  was  an  in-depth  analysis  of  tool  capabilities  for 
the  DBA  BPE  using  a  DBA  scenario  as  a  test-bed  problem.  The 
objective  was  not  to  solve  the  test  problem,  but  to  determine 
metrics  for  tool  comparisons. 

In  Phase  I,  a  tool  survey  was  conducted  with  information 
collected  from  a  number  of  commercial  and  industrial  sources 
and  analyzed  with  respect  to  the  DBA  programming  environment. 
Presentations  and  demonstrations  were  then  conducted  at  each 
center  on  a  set  of  tools  covering  all  aspects  of  the  software 
development  process  during  the  month  of  June  1981.  The  first 
two  weeks  of  June  were  designated  for  presentations  at  DBAHTC 
with  duplicate  presentations  at  DBA AC  the  following  two 
weeks.  There  were  only  minor  differences  between  the  sets  of 
presentations  given  at  the  centers,  relating  to  scheduling 
and  not  to  material  content.  Figure  1.4  shows  the  tool 
presentation  and  demonstration  schedule  for  each  DBA  center. 
Those  tools  outlined  in  cross-hatching  were  presented  by 
their  vendor;  all  other  tools  were  presented  by  G D/DSD 
personnel.  After  each  presentation  a  survey  form  was 
completed  by  the  DBA  attendees  to  evaluate  the  tool  with 
respect  to  applicability  and  appropriateness  to  DBA  needs.  A 
copy  of  this  survey  form  is  included  as  Appendix  B. 

The  tools  were  then  ranked  according  to  perceived  need  and 
applicability  after  the  presentations  by  compiling  statistics 
from  the  survey  form.  The  tool  rankings  are  presented  in 
Figure  1.5  which  is  explained  in  the  following  three 
paragraphs. 

The  left-most  column  is  the  ranking  of  the  tools  based  upon 
the  number  of  survey  responses  with  respect  to  their 
applicability  to  DBA  tasks  and  ease  of  use  in  an  interactive 
environment.  The  higher  a  tool  appears  in  the  list  the 
larger  the  number  of  positive  comments  received.  The 
capabilities  of  some  demonstrated  tools  may  have  been  new  to 
the  evaluators,  and  hence  they  did  not  immediately  perceive  a 
use  for  that  tool.  Therefore,  in  order  to  provide  a  uniform 
baseline  for  ranking  familiar  and  unfamiliar  tool 
capabilities,  the  DBA  survey  responses  were  also  ranked  by 
the  number  of  least  negative  comments  recorded.  The  middle 
column  lists  in  order  the  tools  based  upon  the  number  of 
negative  survey  responses.  Again,  the  best  tool  is  at  the 
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top.  The  lower  a  tool  appears  in  the  list  the  larger  the 
nuaber  of  negative  comments  received.  A  line  was  drawn 
connecting  the  saae  tool  in  the  aost  positive  and  least 
negative  rankings.  The  lore  horizontal  the  slope  of  this 
line  the  higher  the  correlation  between  the  two  ranking 
scheaes.  a  line  with  a  very  steep  slope  indicates  an 
uncertain  correlation.  The  aost  desirable  tools  are  those 
that  are  near  the  top  of  both  lists  and  have  a  high 
correlation  indicated  by  a  nearly  horizontal  line.  Finally, 
the  right-aost  coluan  of  Figure  1.5  is  an  independent 
assessment  by  the  GD/DSD  team  personnel  based  upon  the  most 
positive  ranking  scheme.  These  rankings  were  utilized  in  the 
selection  of  tools  for  Phase  II. 
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In  Phase  II  of  the  tool  evaluation  plan,  a  DBA  software 
scenario  was  simulated  using  a  test-bed  problem  that 
exercised  the  tool  capabilities.  The  test-bed  problem 
utilized  digitized  feature  data  and  a  simulated  cartographic 
data  base  to  extract  a  new  manuscript.  The  software  tools 
were  used  for  requirements  definition,  design,  coding  and 
testing.  In  addition,  a  data  base  management  tool  was  used 
in  the  collection  of  data  from  DBA  team  members  for 
statistical  evaluation;  and  a  prepared  tutorial  on  a  project 
management  tool  was  available  for  viewing.  Figure  1.6 
illustrates  the  schedule  of  activities  as  they  occurred  in 
Phase  II.  Subsequently,  the  DSE.IT  software  tool  was  of 
particular  interest  to  the  BPE  study  participants  because  of 
its  requirements  definition,  design,  and  automated  FORTRAN 
coding  capabilities.  Hence,  it  was  decided  to  evaluate  the 
applicability  of  USE. IT  to  the  DBA  software  environment  by 
solving  a  realistic  DBA  problem.  The  chosen  problem  was  a 
long  range  navigation  (LORAN)  lattice  calculation,  and  the 
evaluation  was  conducted  from  July  to  September,  1982. 
During  this  period  the  LORAN  problem  was  modeled  with  USE. IT, 
executable  code  was  produced,  and  the  graphics  displays  were 
demonstrated  to  DBA.  A  description  of  the  LORAN  problem  is 
included  as  Appendix  I. 

Utilizing  the  data  collected  in  Phase  I  and  Phase  II,  Near- 
Term  and  Far- Term  HPE's  were  developed.  These  recommended 
near-term  and  far-term  environments  meet  the  requirements  as 
specified  in  the  SON/SOC  as  well  as  provide  for  the 
environmental  capabilities  identified  during  the  software 
tool  evaluation.  In  the  Near-Term  BPE  risks  have  been 
minimized  by  recommending  tools  which  are  currently  available 
and  have  been  throughly  investigated  with  respect  to  claimed 
performance  capabilities.  Performance  cannot  be  quantified, 
but  cost  data  and  rationale  are  provided  which  support  our 
conclusions.  An  experimental  system  would  be  developed  first 
in  the  implementation  of  the  environment  to  provide 
engineering  data  to  fine  tune  system  performance.  Further 
information  on  the  experimental  system  can  be  found  in 
Section  19.1. 
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The  near-tera  and  far-term  reconnendations  are  sumnarized  in 
Figures  1.7  and  1.8  respectively.  The  Near-Term  MPE  is  based 
upon  a  VAX  configuration.  This  configuration  provides  a 
software  development  capability  with  minimum  schedule  and 
technical  risk  at  low  cost.  These  systems-  represent  the 
state-of-the-art  in  software  development  tools  when 
constrained  by  DMA's  current  systems  and  future  plans.  The 
Far-Term  MPE  is  also  based  upon  the  VAX  because  of  the 
abundance  of  software  tools  currently  available  and  projected 
to  be  available  for  this  system. 

The  effort  involved  in  the  development  of  the  report  and  its 
associated  annexes  included  over  three  manyears  of  labor  by 
GD/DSD  with  92  mandays  (63  calendar  days)  of  activity  being 
conducted  on-site  at  the  DMA  centers. 

The  acquisition  of  the  Near-Term  MPE  tools  and  tool  bearing 
host  (TBH)  and  its  evolution  to  the  Far-Term  MPE  will  not 
satisfy  all  the  software  development  support  requirements  for 
the  1987  target  date.  The  support  areas  of  cost  estimating, 
management  tools,  tool  set  integration,  code  auditors,  and 
Ada  will  require  research  and  development  activities.  In 
each  area  work  must  be  accomplished  to  define  the  DMA 
specific  needs,  identify  solutions,  and  provide  for  the 
solutions  to  be  integrated  into  the  Far-Term  MPE. 
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TOOLSMITH 


era  System  Conf iquration  for 


2 . o  determination  of.dma  needs 

The  first  stage  of  this  MPE  study  as  stated  in  the  technical 
approach  was  the  determination  of  DMA  needs.  This  included 
the  needs  common  to  both  DMA  centers  and  those  specific  to 
DMAHTC  and  DMAAC.  The  following  list  of  sources  was  used  to 
obtain  data  which  in  turn  was  analyzed  and  the  results 
presented  in  the  SON  described  in  Section  2.3. 

1)  FEDSIM  (Federal  Computer  Performance  Evaluation  and 
Simulation  Center)  Installation  Review  -  DMAHTC  - 
November  1980 

2)  DMA  Operational  Concepts  (1982  -  1990)  -  May  1979 

3)  DMA  Programming  Support  Library  (PSL)  Interim  Evaluation 
Report,  IBM/FSD  -  November  1980 

4)  DMAAC/Scientific  Computer  Division  -  Software  Life  Cycle 
Standards  -  February  1981 

5)  DMAAC  Organizational  Mission  Functions  -  October  1980 

6)  FEDSIM  Installation  Review  -  DMAAC  -  August  1980 

7)  DMA  Modern  Programming  Environment  (MPE)  -  January  1980 

8)  FEDSIM  Optimization  and  Error  Rate  Studies  -  February  1981 

9)  Operational  Improvement  Opportunities  for  DNIVAC  1100/80 

10)  DMAHTC  Organizational  Manual 

11)  General  Dynamics  DMA  Survey  -  March  1981 

12)  Interviews  conducted  by  General  Dynamics  at  DMAHTC  and 
DMAAC 

13)  DMAAC  Modern  Programming  Environment  Pilot  Project 
Evaluation  Report 

14)  The  DMAHTC  Modern  Programming  Environment  (MPE)  Pilot 
Project 

2.1  DMA  SURVEY  QUESTIONNAIRE 

Item  11  of  the  above  list  (see  Appendix  A)  consisted  of  a 
questionnaire  developed  by  General  Dynamics  to  help  determine 
needs  at  DMA  and  help  identify  currently  used  tools 
appropriate  for  common  use.  The  questionnaire  was  also 
planned  to  function  as  a  tool  in  validating  the  findings  of 
the  Boeing  Report,  R ADC-TR-79- 34 3  (item  7,  DMA  MPE  -  January 

1980) ,  as  well  as  a  means  of  gathering  information  about  the 

future  plans  of  DMA  in  the  areas  of  operations  and  policies. 

The  questionnaire  corroborated  the  findings  of  the  Boeing 
Report  with  minor  exceptions  in  the  area  of  project 
management  techniques.  Since  the  Boeing  Report  was 
generated,  DMA  has  started  activities  to  correct  identified 
deficiencies. 

The  questionnaire  consisted  of  five  parts.  The  first  and 

last  sections  were  to  be  answered  by  every  respondent.  The 

first,  the  "respondent"  section,  was  used  to  correlate 
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answers  with  respect  to  a  person's  background.  This  also 
included  questions  to  determine  DBA  organization  (DBAAC  vs 
DHAHTC) ,  environment  (open  vs  closed  shop)  and  security 
(Sensitive  Co^partmented  Information  (SCI)  vs  collateral) 
which  were  to  bo  used  in  the  classification  of  needs.  The 
last,  the  tools  section,  was  included  to  gather  general 
knowledge  about  what  software  tools  exist  at  DBA  and  their 
usefulness.  One  of  each  of  the  three  remaining  sections  was 
to  be  answered  by  each  respondent  according  to  his  job 
classification.  These  included  a  technical  section  to  gather 
data  on  operations,  a  management  section  to  determine  methods 
of  operation  and  a  policies  section  to  be  answered  by  higher 
management  concerning  DBA  planning,  control,  organization  and 
direction. 

230  questionnaires  were  distributed,  10  to  DBAHQ,  110  to 
DBAHTC  and  110  to  DBAAC.  181  were  completed  and  returned, 
43*  from  DBAAC  and  57%  from  DBAHTC.  There  were  28  invalid 
questionnaires  (out  of  the  181)  due  to  one  of  the  following 
not  being  given:  DBA  organization  (DBAAC  vs  DBAHTC) , 
environment  (open  vs  closed  shop)  or  security  (SCI  vs 
collateral) .  No  attempt  was  made  to  validate  these 
questionnaires  with  additional  information  because  the  valid 
sample  size  was  considered  sufficient. 

Data  from  the  DBA  survey  questionnaire  was  collected  and 
stored  in  a  database  inquiry  system  to  be  used  in  compiling 
data  for  the  SON. 

2.2  DBA  PERSONNEL  INTERVIEWS 

The  personnel  interviews,  item  12,  were  conducted  during 
Barch,  April  and  Hay,  1981  with  representatives  at  each 
center  and  were  used  to  gain  additional  insights  into  DBA 
center  activities  and  to  gather  supporting  information  on 
their  needs. 

2.3  STATEBENT  OF  OPERATION  NEEDS  (SON) 

The  resulting  SON  (Figure  2.1)  has  three  major  columns,  BASIC 
DATA  and  NORHALIZED  DATA  by  area  (working  environment)  and  a 
list  of  DBA  needs. 

The  basic  data  represents  actual  responses  to  database 
inquiries  (for  numeric  data)  and  annotations  from  manuals  in 
which  needs  were  presented  (alphanumeric  data) .  Note  that 
needs  identified  in  associated  manuals  were  included  only  if 
a  similar  category  was  not  present  in  the  survey  responses 
(numeric  data) . 
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The  normalized  data  resulted  from  the  fact  that  the  number  of 
responses  in  any  particular  area  were  different  from  the 
number  of  responses  in  the  other  areas.  To  form  a  common 
base  the  numeric  basic  data  was  normalized  by  forming 
percentages  with  the  total  number  of  responses  addressing  a 
particular  need. 

Within  the  basic  and  normalized  columns  the  data  was  broken 
out  additionally  by  common  management,  closed  shop  non- 
secure,  closed  shop  secure,  open  shop  non-secure  and  open 
shop  secure  using  the  respondent  background  information 
obtained  in  the  first  section  of  the  questionnaire  and 
according  to  the  following  definitions: 

1)  Common  management  represents  common  needs  across  all  shops  as 
perceived  by  management. 

2)  Secure  vs  non-secure  is  the  same  as  SCI  vs  collateral. 

3)  Open  shop  meant  the  computer  could  be  accessed  by  all 
qualified  individuals. 

4)  Closed  shop  meant  a  restricted  staff  was  assigned  for 
computer  use  and  operation. 
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Figure  2. 1 


3 . 0  SON  VALIDATION 


During  the  aonth  of  June,  1981  meetings  were  held  at  DMAHTC 
and  DMAAC  to  validate  the  findings  presented  in  the  SON 
(Figure  2.1). 

The  first  aeeting  was  held  at  DMAHTC  with  General  Dynamics 
Central  Center  {DSD/Central  Center)  project  personnel  and  two 
DMAHTC  techniques  office  representatives.  At  this  aeeting 
the  SON  was  presented  using  the  normalized  data  given  in 
high,  medium  and  low  format,  where  high  meant  68-99%  of  those 
responding  saw  a  need,  medium  meant  34-67%  and  low  meant  0- 
33%.  Discussion  topics  were  open  vs  closed  shop,  secure  vs 
non-secure,  continuation  meetings  with  organizational 
representatives,  and  alternatives  to  the  SON  breakouts  (open 
shop,  closed  shop,  secure,  non-secure) .  No  consensus  was 
reached  due  to  ambiguities  in  the  definitions  of  "access", 
"open  shop"  and  "closed  shop". 

A  second  meeting  at  DMAHTC  was  attended  by  the  DSD/Central 
Center  project  personnel  and  management  representatives  from 
multiple  organizations.  It  was  first  decided  that  no 
concensus  could  be  determined  for  definitions  of  open  vs 
closed  shop  or  for  an  alternative  breakout  (minicomputers  vs 
mainframe  for  example) .  Therefore  the  breakouts  were 
eliminated.  Next  the  meanings  of  the  needs  and  their 
applicability  to  DMA  were  defined.  Several  needs  were 
discarded  as  they  were  covered  by  larger  categories  in  the 
list.  Finally  all  the  needs  identified  were  rated  as  high, 
medium  or  low  needs. 

A  third  aeeting  was  held  at  DMAHTC  with  the  DSD/Central 
Center  project  personnel  and  technical  representatives  from 
various  organizations.  Starting  with  the  list  of  needs  as 
set  in  the  management  aeeting  the  needs  were  again  defined 
and  rated. 

The  SON  meetinq  at  DMAAC  was  attended  by  the  DSD/Central 
Center  project  personnel  and  four  DMAAC  representatives  from 
multiple  organizations.  The  list  of  needs  was  presented  as 
developed  at  DMAHTC  and  again  defined  and  rated  as  high, 
medium  or  low  needs. 

Composites  were  made  of  the  ratings  by  center  and  included 
inputs  from  DSD/Central  Center  project  personnel  (see  Figures 
3.1  and  3.2).  The  following  rating  scheme  was  used  in  the 
resulting  revised  SON  (Figure  3.3): 

5  =  high 

4  =•  medium  high 
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3  =  medium 
2  =  medium  low 
1  =  low 


The  needs  referenced  in  governaent  documents,  needs  42-59, 
were  rated  high  if  the  need  was  called  out  in  aore  than  one 
document  and  medium  if  it  was  mentioned  in  only  one.  A  blank 
in  the  rating  column  indicates  that  there  was  no  need 
identified  in  that  category  for  a  particular  DBA  center.  The 
extreme  left-hand  column  of  numbers  serves  to  provide  numeric 
reference  and  tracking  to  the  original  SON  for  each  need; 
omissions  in  the  sequence  occur.  For  example,  some  were 
eliminated  from  the  SON  (see  F^iure  5.1)  because  they  were 
covered  by  larger  categories  in  the  list 
(8,13,19,20,23,29,31,32,35,37,38,39,50),  were  too  broad 
(15,17,30)  or  were  outside  the  scope  of  the  study 
(25,43,45,51,53). 


Need 

Number  (s) 


Incorporated 
Into  Need  (s) 


6,23 

8 

13 

19,20 

28 

29 

31,32 

35 

37,38,39 

49 


59 

1,22,59 

37 

1 

34 

9 

57 

22 

56 

42 


Need  number  60  was  added  during  DMA  HPE  study  team  meetings 
at  DSD/Central  Center  in  Fort  Worth  in  July  1981  while 
refining  the  SON/SOC  (System  Operational  Concept)  matrix 
which  will  be  discussed  next;  and  was  a  redefinition  of 
number  7.  For  needs  24,26,27  and  33  additional  study 
determined  there  was  no  actual  need  currently  or  projected  at 
DMA. 
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Figure  3.3 
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4 . 0  SYSTEM_0  PE  RATIO  NAL_CONCEPTS _ (,SOC)_ 


1 
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The  second  stage  of  this  modern  programming  environment  study 
as  stated  in  the  technical  approach  was  the  formulation  of 
system  concepts  to  satisfy  those  needs  identified  in  the  SON. 
The  SOC  is  a  list  of  concepts  covering  such  areas  as 
hardware,  software,  methodologies,  training  and  support  that 
form  the  basis  for  a  modern  programming  environment  for  DMA. 
The  SOC  is  the  hey  link  between  the  SON  and  the  particular 
implementation  to  be  proposed  for  a  Near-Term  (1985)  and  Far- 
Term  (1987)  MPE. 

The  SOC's  were  formulated  from  the  needs  identified  in  the 
SON,  MPE  concepts,  and  GD/DSD  experience  during  project 
personnel  team  meetings  at  DSD/Central  Center  in  Fort  Worth 
during  the  month  of  July,  1981.  The  SON/SOC  matrix  (see 
Figure  5.2)  resulting  from  these  meetings  is  a  mapping  of  the 
operational  needs  identified  in  the  SON  into  one  or  more 
generic  programming  concepts  which  satisfy  each  need. 

Needs  were  identified  as  center  specific  when  grouping  showed 
a  high  need  at  one  center  and  a  low  need  at  the  other.  All 
needs  with  a  low  rating  at  both  sites  inclusively  have  been 
eliminated  from  the  SON/SOC.*"  The  columns  of  the  matrix 
represent  various  concepts  that  could  satisfy  the  particular 
needs.  When  a  need  is  partially  or  completely  satisfied  by  a 
concept  an  "X"  appears  at  the  point  of  intersection  in  the 
matrix.  Note  that  a  particular  concept  can  satisfy  more  than 
one  need  and  a  particular  need  may  require  more  than  one 
concept  to  satisfy  it. 

The  SON/SOC  matrix  has  two  outstanding  benefits.  Consistency 
can  be  traced  between  the  SON  and  the  SOC,  and  the  system 
concepts  are  generic  in  nature  which  allows  for  more  than  one 
implementation  method.  The  proposed  alternative 
implementations  were  to  be  used  in-part  to  develop  the  Near- 
Term  and  Far-Term  Modern  Programming  Environment 
specifications  for  DMA. 


5 • o  IN- PROCESS- RE VIEW  OF  SON/SOC 

The  SON/SOC  matrix  was  presented  at  an  In-Process-Review 
(IPR)  conducted  at  RADC  on  18-19  August,  1981.  As  a  result 
of  this  review  several  changes  were  made  to  the  SON/SOC 
matrix  and  the  SON.  The  changes  involved  elimination  of 
certain  concepts  (20-automated  tape  management  procedures, 
23-software  scheduling  package  and  26-information 
interchange)  and  needs  {43-better  tape  procedures,  45- 
replace/terminate  DCT  2000's  and  51-remote  access  by  federal 
agencies)  as  being  outside  the  realm  of  the  DMA  MPE  study 
since  they  were  not  part  of  software  development.  In 
addition  SOC11  was  revised  to  apply  more  closely  to  current 
DMA  needs  and  S0C18  was  expanded  to  include  graphics.  It  was 
also  requested  that  the  needs  be  categorized  in  some  manner 
to  improve  on  the  readability  of  the  matrix.  This  was 
accomplished  by  grouping  the  needs  relative  to  the  software 
life  cycle;  with  some  needs  appearing  in  multiple  groups. 
The  revised  SON  and  SON/SOC  appear  in  Figures  5.1  and  5.2 
respectively. 

At  a  follow-up  status  meeting  in  St.  Louis,  certain  center 
specific  needs  were  eliminated  because  of  an  improved 
understanding  of  the  purpose  of  a  "DMA  need".  It  was  decided 
that  the  needs  should  reflect  MPE  needs,  not  necessarily 
center  needs.  Hence,  certain  needs  that  were  center  specific 
were  eliminated  from  the  SON. 
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Figure  5.1  Current  SON 
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6.0  MEEDS 


The  following  paragraphs  define  in  detail  the  needs  as  listed 
in  the  revised  SON  (Figure  5.1)  and  their  applicability  to 
DM1.  First  a  definition  of  the  need,  which  nay  include 
amplification  with  respect  to  the  DMA  environment,  is  given 
followed  by  a  list  of  documents  in  which  the  needs  were 
identified.  For  further  identification  of  related  documents, 
the  number  in  parenthesis  refers  back  to  the  list  in  Section 
2.0.  Page  and  paragraph  numbers  are  included. 
Implementation  priority  is  then  provided. 

6.1  DEFINITION  AND  ORIGIN  OF  NEEDS 


The  following  items  were  found  to  be  medium  to  high  needs  at 
both  DMAHTC  and  DMAAC.  First  the  need  is  defined,  then  the 
origin  is  given.  The  need  was  identified  through  supporting 
DMA  documentation,  through  interviews  with  DMA  personnel  or 
through  a  survey  questionnaire  distributed  to  management  and 
technical  personnel  at  both  centers. 


6.1.1  (SON # 1 )  Formal  Requirements  Specification:  a  means  of 
formally  documenting  the  elemental  requirements  of  a  task  or 
project  prior  to  the  beginning  of  design.  The  method  used 
may  be  a  manual  or  automated  method  involving  the  use  of  a 
software  requirements  tool.  Examples  of  these  tools  are 
USE. IT,  SADT,  PSL/PSA ,  LIRE  and  FAME. 


QEIGINlPAGES 

DMA  MPE  STUDY  (7)  :  30,36 

GD  SURVEY/QUESTIONS  (1 1)  :  I.1.B,  I.1.R,  I.2.H,  1. 2. 1, 
I.2.J,  I.2.R,  I.2.S,  III.G 
Personnel  Interviews  by  GD/DSD(12) 


6.1.2  (S0N#2)  Quality  Assurance  Procedures  and  Guidelines: 
ways  of  enforcing  a  required  set  of  programming 
practices/standards  covering  all  phases  of  the  pro7ramming 
lifecycle.  This  provides  for  both  better  quality  and 
consistency  in  software  development  and,  therefore,  more 
easily  maintained  software. 


ORIGINiPAGES 

FEDS  I M  REVIEW  -  DMAHTC  ( 1)  :  19,32 

PROGRAM  SUPPORT  LIBRARY  -  INTERIM  REP0RT(3):  2-4 
DMA  MPE  STUDY  (7)  :  36-40 

GD  SURVEY/QUESTIONS  (11)  :  I.2.P,  I.2.R,  II. 2. C 
Personnel  Interviews  by  GD/DSD(12) 
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6.1.3  (SON# 3)  Interactive  System  Access:  the  ability  to 
access  the  computer  system  through  an  on-line  environment  as 
opposed  to  card  readers  or  over-the-counter  entry  stations. 
Output  may  also  be  accessed  without  necessarily  being  printed 
out  on  paper.  This  results  in  more  freedom  of  access,  faster 
turnaround  time  and  a  decrease  in  the  amount  of  paper 
produced. 

ORIGINlP AGES 

DMA  OPERATIONAL  CONCEPTS  (2) :  3,  31 

DMA  HPE  STUDY (7) :  47 

GD  SURVEY/QUESTIONS  (11)  :  1.1. A,  I.1.C,  1.1. F,  I.1.G, 

I.1.T,  I. 2. L, IV. A 

Personnel  Interviews  by  GD/DSD(12) 

6.1.4  (S0N#4)  An  Increased  Number  of  Terminals:  a  requirement 
at  DMA  centers  if  interactive  access  is  to  be  made  available 
to  all  programmers.  Currently  there  are  a  minimal  number  of 
terminals  available  through  which  the  programmers  may  obtain 
this  access. 

ORIGIN : PAGES 

Personnel  Interviews  by  GD/DSD(12) 

6.1.5  (S0N#5)  Requirements  Tracking:  a  means  of  documenting 
the  coverage  of  and  changes  to  the  requirements  of  a  program 
or  system  through  its  complete  lifecycle.  As  with 
requirements  specification  this  may  be  done  through  a 
standardized  manual  method  or  a  commonly  used  automated 
method  (software  tool) . 

ORIGIN : PAGES 

GD  SURVEY/QUESTIONS  (1 1)  :  I.1.B,  I.2.F,  I.2.H,  I.2.R, 

I.2.S 

8.1.6  (SON# 9)  Configuration  Control:  the  ability  to  track  and 
maintain  a  history  of  changes  to  a  system  or  program.  Within 
DMA  programs  are  commonly  sent  between  organizations  or 
centers.  As  changes  occur  to  these  programs  they  are  not 
necessarily  made  to  all  production  versions  and  eventually 
the  program  may  no  longer  be  a  common  system  to  all  users.  A 
configuration  control  system  would  keep  track  of  these 
versions  by  the  use  of  version  or  release  numbers  and 
maintain  a  history  of  the  changes  required  to  get  from  one 
version  to  another;  thus  improving  communications  between 
users  of  a  system  and  providing  consistency  in  the  use  of  a 
system. 
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QRIGINJ.PAGES 

PROGRA  M~ SUPPORT  LIBRARY  -  INTERIM  REPORT  (3) :  2-8 
DMA  MPE  STUDY  (7) :  48-58 

GD  SURVEY/QUESTIONS  (11)  :  I.1.C,  I.1.L,  I.2.B,  II.1.C, 
II.1.H,  IV. A 

6.1.7  (SON#10)  Improved  Milestone  Identification:  a  leans  of 
improving  the  identification  and  documentation  of  significant 
events  in  the  development  of  a  system  or  program.  This 
provides  an  overall,  high  level  view  of  a  system's 
development  process. 

QRIGIN:PAGES 

GD~SURVEY/QUESTIONS  (11)  :  1. 2. A,  I.2.G,  I.2.H,  II.1.D 
Personnel  Interviews  by  GD/DSD(12) 

6.1.8  (SON# 11)  Decreased  Paperwork:  a  need  to  lover  the 
amount  of  paperwork  produced  at  each  center  intruding 
computer  runs  and  manually  produced  documentation  related 
with  software  development. 


ORIGIN ; PAGES 

GD~SUR V£ Y/QUESTION S  ( 1 1 )  :  I. 1. 3,  I.2.H,  II.1.G,  II. 2. A, 
II. 2. B,  III. H 

Personnel  Interviews  by  GD/DSD(12) 


6.1.9  (SON# 1 2)  Improve  Manloading:  improvement  of  the  methods 
of  determining  the  amount  of  manpower  reguired  for  a  given 
project  through  manual  or  automated  methods  using  parametric 
or  historical  data. 


QlIGINiPAGES 

Personnel  Interviews  by  GD/DSD(12) 

GD  SURVEY/QUESTIONS  (11)  :  I.1.H,  I.2.E,  II.1.D,  IV. A 

6.1.10  (SON# 1 4)  improve  Schedule  Impact  Analysis:  improvement 
of  the  methods  of  determining  how  changes  to  a  project  will 
affect  its  schedule  through  automated  or  manual  methods 
usually  associated  with  identifying  critical  and  affected 
paths  in  the  development  process. 

ORIGIN : PAGES 

GD~SUR VEY/QUESTIONS  (11)  :  I.2.C,  I.2.D,  II.1.D,  IV. A 
Personnel  Interviews  by  GD/DSD(12) 

6.1.11  (SON#  16)  Update  of  old  Documentation:  improvement  of 
the  documentation  associated  with  existing  programs  available 
for  maintenance  purposes. 

OUGINiPAGES 
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Personnel  Interviews  by  GD/DSD(12) 


6.1.12  (S0N#18)  Faster  Integration  of  New  Employees:  to 
provide  assistance  in  the  training  and  orientation  of  new 
employees  into  the  programming  environment  of  DMA  and  its 
associated  standards  and  methodology  for  software 
development. 


ORIGIN : PAGES 

Personnel  Interviews  by  GD/DSD(12) 

6.1.13  (S0NI21 )  Simulator  for  Design:  a  system  of  software 
tools  which  would  enable  the  rapid  prototyping  of  a 
production  environment  in  order  to  verify  the  basic  design  of 
developed  software. 

ORIGINiPAGES 

GD~SURVEY/QOESTIONS  (1 1) :  IV. A 

6.1.14  (SON#22)  Program  Design  Language:  a  language  used  in 
the  design  and  documentation  of  complex  software 
applications. 

ORIGIN: PAGES 

GD  SDR VE Y/QDESTI ONS  ( 1 1 )  :  I.2.K,  I.2.S,  IV. A 

6.1.15  (SON #34)  Automated  Text  Management  System:  a  software 
system  which  would  provide  basic  support  in  the  development 
of  textual  material  associated  with  the  software  development 
process  including  such  functions  as  sorting,  merging, 
copying,  formatting  and  archiving;  as  well  as  the 
capabilities  associated  with  text  editing  tools. 

QBIGINiPAGES 

DMA  OPERATIONAL  CONCEPTS  (2)  :  8 
GD  SDRVEY/QDESTIONS  (11)  :  IV. A 
Personnel  Interviews  by  GD/DSD(12) 

6.1.16  (SON#36)  Graphics  Aids:  hardware  and/or  software  which 
would  provide  the  capability  to  display  plotter  type 
information  in  an  interactive  CRT  format  or  change  data  from 
one  format  to  another  for  display. 

ORIGINiPAGES 

GD~ SURVEY/QUESTIONS  (1 1)  :  IV. A 
Personnel  Interviews  by  GD/DSD(12) 

6.1.17  (SON#40)  Historical  Data  Base  Techniques:  methods, 
either  manual  or  automated,  of  collecting  information  and 
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statistics  associated  with  software  development  activities  to 
provide  a  basis  for  evaluation  of  future  tasks. 

ORIGIN  jPAGES 

GD~SORVEY/QUESTIONS(11) :  I.2.C,  IV. A 

6.1.18  (S0N#4 1 )  Organizational  Tools/Techniques  Interface:  a 
■eans  of  providing  a  common  format  for  the  exchange  of  ideas 
and  information  between  organizations  within  DMA  which  have  a 
functional  dependency. 

QBIGINiPAGES 

GD  SURVEY/QUESTIONS  (11)  :  IV. A 

Personnel  Interviews  by  GD/DSD(12) 

6.1.19  (S0N#42)  User  Assistance  Punction:  a  method  to  assist 
users  in  overcoming  and  avoiding  errors.  The  user  assistance 
function  people  would  not  be  expected  to  help  users  debug 
their  programs  but  would  help  all  users  who  had  production 
run  problems.  Additional  duties  would  be  conducting  error 
rate  studies,  conducting  meetings  with  users  explaining  how 
to  avoid  errors,  disseminating  information  on  the  better  use 
of  the  computer  system,  including  a  system  change  bulletin, 
and  augmenting  the  information  flow  to  management  so  they  may 
respond  more  quickly  to  user  needs. 

ORIGIN iPAGES 

FEDSIM  REVIEW  -  DMAHTC { 1 ) :  24-25 

FEDSIM  REVIEW  -  DMA AC  (6) :  37,  49-50 

FEDSIM  ERROR  RATE  STUDIES (8) :  42 

6.1.20  (S0N§44)  Error  Rate  Standards:  the  formulation  of 
limits  on  specific  error  repetitions,  possibly  within  a  given 
time  frame.  Reports  would  be  generated  on  these  errors  and 
sent  to  all  organizations.  Corrective  action  can  then  be 
taken  by  each  organization  for  areas  where  limits  are 
exceeded  or  justification  provided  for  exceeding  the  limit. 
A  method  of  revising  the  limits  must  be  included  in  the 
standards. 

QRIGINiPAGES 

DMA  MPE~STUDY  (7)  :  43-45,  47 

FEDSIM  ERROR  RATE  STUDIES (8) :  42-43 

6.1.21  (SON#46)  Reduced  Accounting  Data  Report  Anomalies: 
DMA's  accounting  data  is  a  conservative  indicator  of  the 
overall  error  rate.  The  accounting  file  reports  more 
erroneous  runs  as  good  than  good  runs  as  erroneous.  These 
erroneous  reports  should  be  reduced  in  number. 
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QRIGI N j  PAGES 

FEDSIM  ERROR  RATE  STUDIES  (8) :  41-42 

6.1.22  (SON#47)  Comprehensive  Training  Program:  training  to 

provide  personnel  with  a  background  in  software  development 
techniques,  requirements  specification,  design,  testing, 
standard  practices,  project  planning,  estimating  and 
scheduling. 


QSIginxpages 

FEDSIM  REVIEW  -  DMAHTC(I):  24,  26-27,  33 
DMA  OPERATIONAL  CONCEPTS  (2)  :  25 

PROGRAM  SUPPORT  LIBRARY  -  INTERIM  REPORT  (3) :  2-14,  2-15 
FEDSIM  REVIEW  -  DMA AC (6) :  29-31,  59 
DMA  MPE  STUDY  (7)  :  58-71,  82-95 
GD  SURVEY/QUESTIONS  (11)  :  II. 2. A,  III.F 

6.1.23  (SON# 48 )  A  Chargeback  System:  a  system  which  assigns 
charges  to  each  unit  of  computer  usage  by  user  and  by  run 
such  that  each  user  run  has  a  unit  charge  associated  with  it. 

The  real  aim  of  the  system  is  not  so  much  to  allocate  costs 
as  to  create  the  proper  incentives  for  the  users  to  become 
involved  in  ADP  management  and  to  conserve  their  use  (i.e., 
use  fewer  tapes,  run  fewer  jobs,  and  make  those  jobs  more 
efficient)  to  enable  the  computer  facility  to  provide 
responsive,  efficient  service. 


QSI^IN i£*GES 

FEDSIM  REVIEW  -  DMAHTC(I):  14,  31 
FEDSIM  REVIEW  -  DMAAC  (6)  :  24-25,  54 

6.1.24  (S0N#52)  To  Decrease  Turnaround  Time  to  Minutes:  both 

centers  are  experiencing  lengthy  turnaround  times  on  the 
mainframe  computers.  A  decrease  in  turnaround  time  to 
minutes  for  the  predominate  number  of  runs  will  be  required. 
Currently  next  day  batch  service  is  normal. 


ORIGINiPAGES 

DMA  OPERATIONAL  CONCEPTS (2) :  58 

PROGRAM  SUPPORT  LIBRARY  -  INTERIM  REPORT  (3)  :  2-4 
FEDSIM  REVIEW  -  DMAAC  (6)  :  37 
DMA  MPE  STUDY  (7)  :  43,  47 
GD  SURVEY/QUESTIONS  (11) :  I. 1.1,  II. 2. A 

6.1.25  (SON#54)  Natural  Language  User/System  Interface:  a 

system  which  would  interface  the  user  to  the  computer  in  a 
manner  which  is  less  constrained  in  syntax  and  semantics  than 
normal  control  and  algorithmic  languages. 


ORIGINiPAGES 


58 


* 


DMA  OPERATIONAL  CONCEPTS  (2)  :  13,  31 


6.1.26  (SON#55)  Modern  Source  Data  Entry  Techniques:  the 
capability  to  enter  data  into  a  computer  through  the  aost 
efficient  means  available  matching  the  fora  of  the  data  to  be 
entered,  for  example,  disk,  floppy,  source,  binary,  cards, 
tape  and  the  systems  available,  i.e.,  CRT,  RJE,  card  reader, 
disk/tape  drive,  etc. 

QSI£IHiP£GES 

DMA  OPERATIONAL  CONCEPTS  (2)  :  34 

6.1.27  (SON#56)  Management  Tracking  Functions:  processes 
available  to  project  managers  which  provide  cost  analysis, 
budget  tracking,  schedule  impact  information  and  report 
generation  capabilities. 

QRIGINiPAGES 

FiDSIM  REVIEW  -  DMAHTC(I):  12 

DMA  OPERATIONAL  CONCEPTS  (2)  :  32 

PROGRAM  SUPPORT  LIBRARY  -  INTERIM  REPORT  (3):  2-8 

DMA  MPE  STUDY  (7)  :  48 

6.1.28  (SON#57)  Software  Development  Tools:  computer  programs 
which  enable  the  user  to  perform  activities  in  the  life  cycle 
development  of  software  without  extensive  training  and/or 
which  decrease  the  amount  of  manual  labor  associated  with  the 
activity;  an  example  being  one  high  order  language  for 
several  computers.  High  order  languages  are  easier  to  learn 
than  assembly  languages  and  may  be  common  to  several 
architectures.  Most  of  the  tools  will  guide  a  person  through 
the  steps  of  a  process;  hence  extensive  training  in  an  area 
such  as  requirements  specification  would  not  be  required.  At 
the  same  time  most  tools  provide  automatic  documentation  and 
analysis  of  its  task. 

SEIginipages 

DMA  OPERATIONAL  CONCEPTS  (2)  :  31 

PROGRAM  SUPPORT  LIBRARY  -  INTERIM  REPORT  (3) :  2-10 

DMA  MPE  STUDY  (7) :  71-78 

GD  SURVEY/QUESTIONS  (11)  :  I.2.L,  IV. A 

6.1.29  (SON# 58)  Production  Program  Optimization:  optimization 
of  programs  which  have  high  computer  resource  requirements 
through  the  use  of  automated  tools  to  identify  structures  or 
code  which  could  be  modified  to  decrease  the  programs  effects 
on  the  production  environment  resources. 

QRIGINiPAGES 

FiDSIM  REVIEW  -  DMAHTC(I):  18,  33 
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PROGRAM  SUPPORT  LIBRARY  -  INTERIM  REPORT  (3) :  2-5 

DMA  MPE  STUDY  (7):  42-47 

FEDSIM  ERROR  RATE  STUDIES  (8) :  41 

6.1.30  (SON#59)  Standardized  Phased  Development:  development 
of  software  in  a  life  cycle  phased  methodology  consistently 
across  the  DMA  organization.  This  would  include 
standardization  of  programming  tools  and  techniques, 
documentation  and  configuration  control. 

ORIGIN : PAG  ES 

Personnel  Interviews  by  GD/DSD112) 

6.1.31  (SONI60)  Standardized  Development  Hardware:  common 
development  hardware  used  throughout  DMA  to  maximize  the 
portability  of  development  tools,  increase  the  efficiency  of 
any  configuration  control  system  and  decrease  the  training 
required  by  the  use  of  diversified  architectures. 

QRIGINiPAGES 

PROGRAM  SUPPORT  LIBRARY  -  INTERIM  REPORT(3):  2-8,  2-9, 

2-  10 

FEDSIM  REVIEW  -  DMAAC (6) :  47,  60 
6.2  PRIORITY  OF  NEEDS 

This  grouping  of  needs  is  a  priority  list  of  the  needs  which 
are  expressed  in  the  SON.  The  first  group  has  the  highest 
priority,  the  last  group  the  least.  The  data  used  to  develop 
this  list  includes  information  gathered  during  the  tool 
evaluation  phase,  October  -  November,  1981;  general  knowledge 
of  the  DMA  environment;  and  the  need  for  a  smooth  transition 
during  implementation  of  solutions.  Rationale  is  included 
for  the  grouping  of  needs  generated  by  GD/DSD,  DMAHTC  and 
DMAAC.  The  needs  given  the  highest  priority  will  be  those 
addressed  first  when  transitioning  from  the  current  DMA 
environment  to  the  near-term  and  subsequent  far-term 
environments.  Implementing,  as  possible,  the  solutions  to 
the  highest  ranked  priorities  first  will  result  in  more 
immediately  apparent  benefits  during  the  implementation  of 
the  MPE.  Additionally,  use  of  the  rankings  will  assure  the 
most  thorough  coverage  within  the  recommended  MPE  of  the  most 
critical  needs  within  DMA. 

6.2.1  Comparison  by  Group 

First  groups  are  identified,  followed  by  the  needs  which  an 
organization  perceived  to  fall  into  the  categories.  The  data 
following  paraphrases  the  rationale  by  which  the  organization 
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prioritized  the  needs.  The  last  set  of  grouped  needs  is  a 
numerical  average  of  the  five  rankings  submitted. 

GD/DSD  GROUPING 

GROUP  5:  3,4 

GROUP  4:  52,55,58,60 

GROUP  3:  1,2,9,22,34,41,48,57,59 

GROUP  2:  5, 10, 1?, 14, 18,42,44,46,47,56 

GROUP  1:  11,16,21,36,40,54, 

Group  5  --  provide  users  with  an  interactive  access  capability  to 
current  development  hardware. 

Group  4  --  provide  the  users  with  a  near-term  hardware/software 
support  system  consisting  of  immediately  available 
tools. 

Group  3  —  integrate  and  modified  as  necessary  to  provide  a 

consistant  and  environmentally  compatible  methodology 
for  software  development. 

Group  2  —  a  partially  parallel  effort  to  provide  training  and 
management  support  must  be  implemented. 

Group  1  --  advanced  support  systems  should  be  provided  and  old 
systems  upgraded  or  replaced. 

I2!!4A£_£Eouping 
GROUP  5:  3,4 

GROUP  4:  22,34,36,41,42,47,52,55,57,58,60 

GROUP  3:  1,2,9,10,11,12,14,40,44,46,48,56,59 

GROUP  2:  5,16,18,21 

GROUP  1:  54 

Group  5  —  increase  programmer  productivity  by  providing  guicker 
access  and  improved  response  time. 

Group  4  —  provide  hardware/software  system  support,  tool 
integration,  and  standards. 

Group  3  —  provide  advanced  automated  management  support  tools. 
Group  2  —  provide  support  for  old  programs  and  new  developments. 
Group  1  —  analyze  advanced  technigues/capabilities. 

DHAHTC  GROUPING 

GROUP  5:  3,4,52,55 

GROUP  4:  2,9,16,22,34,36,41,42,57,58,59 

GROUP  3:  10,12,14,18,44,46,47,48,56 

GROUP  2:  1,5,21,40 

GROUP  1:  11,54,60 

i 

Group  5  --  the  need  to  improve  programmer  productivity  and  user 
access. 


Group  4  —  the  need  to  improve  management  of  software  projects 
through  establishment  of  standards,  tools  and 
procedures;  software/hardware  support. 

Group  3  --  the  need  to  improve  general  administrative  management 
through  better  resource  allocation,  scheduling  and 
accounting . 

Group  2  --  the  need  to  provide  the  capability  for  system 
definition  and  design. 

Group  1  —  the  need  for  advanced  software/technical  support. 
DHAH2_GaOUPING 

GROUP  5:  2,3,4,9,40,47,48,55,56,57,59,60 

GROUP  4:  12,14,16,36,41,42,44,58 

GROUP  3:  1,5,10,11,18,22,34,46,52 

GROUP  2:  21 

GROUP  1:  54 

RADC.GROUPING 

GROUP  5:  3,9,34,47,55,57,60 

GROUP  4:  10,22,36,42,58 

GROUP  3:  2,4,16,46,48,54,56,59 

GROUP  2:  1,5,11,  12,40,41 

GROUP  1:  14,18,21,44,52 

AVERAGE_GROUPING 

GROUP  5:  (4. 3-5.0)  3,4,55 

GROUP  4:  (3. 5-4. 2)  2,9,22,34,42,47,57,58,59,60 

GROUP  3:  (2.7-3. 4)  1,10,12,16,36,41,46,48,52,56 

GROUP  2:  (1.9-2. 6)  5,11,14,18,40,44 

GROUP  1:  (1.0-1. 8)  21,54 

6.2.2  Comparison  by  Need 

GD/DSD,  RADC ,  DHAHQ,  DM A AC  and  DMAHTC  personnel  supplied 
inputs  to  help  define  a  priority  weighting  factor  for  each 
need  from  the  SON.  These  are  on  a  scale  of  5-1  with  a  5 

indicating  the  greatest  need  and  1  the  least.  These 

weighting  factors  were  used  in  computing  the  total  score  for 
an  implementation  of  a  concept  as  described  in  Section  13.0. 

ABBREVIATIONS:  DSD-G, DM A AC- A , DMAHTC-H, RADC-R, DMAHQ-Q 

G  A  H  R  Q  avg. 

1  FORMAT.  REQUIREMENTS  SPECIFICATION  3  3  2  2  3  2.6 

2  QA  PROCEDURES  AND  GUIDELINES  33435  3.6 

3  INTERACTIVE  SYSTEM  ACCESS  55555  5.0 

4  INCREASED  NUMBER  OF  TERMINALS  55535  4.6 


5  REQUIREMENTS  TRACKING 

9  CONFIGURATION  CONTROL 

10  IMPROVE  MILESTONE  IDENTIFICATION 

11  DECREASED  PAPERWORK 

12  IMPROVE  MANLOADING 

19  IMPROVE  SCHEDULE  IMPACT  ANALYSIS 
16  UPDATE  OF  OLD  DOCUMENTATION 
18  FASTER  INTEGRATION  OF  NEW  EMPLOYEES 

21  SIMULATOR  FOR  DESIGN 

22  PROGRAM  DESIGN  LANGUAGE 

34  AUTOMATED  TEXT  MANAGEMENT  TOOL 
36  GRAPHICS  AIDS 

40  HISTORICAL  DATA  BASE  TECHNIQUES 

41  ORGANIZATIONAL  TOOLS/TECHNIQUES  INTERFACE 

42  USER  ASSISTANCE  FUNCTION 
44  ERROR  RATE  STANDARDS 

46  REDUCE  ACCOUNTING  DATA  REPORT  ANOMALIES 

47  COMPREHENSIVE  TRAINING  PROGRAM 

48  CHARGEBACK  SYSTEM 

52  DECREASE  TURNAROUND  TIME  TO  MINUTES 

54  NATURAL  LANGUAGE  USER/SYSTEM  INTERFACE 

55  MODERN  SOURCE  DATA  ENTRY  TECHNIQUES 

56  MANAGEMENT  TRACKING  FUNCTIONS 

57  SOFTWARE  DEVELOPMENT  TOOLS 

58  PRODUCTION  PROGRAM  OPTIMIZATION 

59  STANDARDIZED  PHASED  DEVELOPMENT 

60  STANDARIZED  DEVELOPMENT  HARDWARE 
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7.0  DEFINITION  OF  CONCEPTS 


In  the  following  subparagraphs  each  of  the  generic  concepts 
identified  in  the  SON/SOC  matrix  is  defined  and  rationale  is 
provided  for  satisfying  the  indicated  needs. 

7.1  SOC  1  INTEGRATED  SUPPORT  DEVELOPMENT  SYSTEM 

DEFINITION:  An  integrated  set  of  tools  developed  to  support 
all  or  part  of  the  software  development  life  cycle,  i.e., 
reguirements,  design,  coding,  testing,  and  maintenance;  and 
to  support  management  functions,  usually  written  in  the 
language  supported. 

SON  ITEMS  SUPPORTED: 1,9, 11, 16,34,36,56,57,58,59 

An  integrated  set  of  tools  supporting  the  entire  life  cycle 
of  software  development  would  include  a  methodology  which 
could  be  formalized  as  a  DMA  standard.  A  configuration 
control  subsystem  would  be  an  integral  part  of  the  life  cycle 
toolset.  Other  tools  in  the  system  would  include  an 
automated  text  management  system  to  support  programming  and 
graphics  aids  to  support  testing.  These  other  tools  would 
also  support  project  management  functions.  Paperwork  would 
be  decreased  due  to  life  cycle  phases  being  supported 
interactively  as  opposed  to  manually;  old  documentation  could 
be  updated  by  processing  old  programs  through  tools  which 
provide  documentation  as  part  of  their  outputs.  These 
outputs  would  also  be  very  useful  in  optimizing  production 
programs  by  identifying  current  capabilities  and  complexities 
associated  with  their  execution.  All  of  the  tools  and 
techniques  for  utilization  could  be  adopted  or  molded  to 
conform  to  a  standardized  phased  development  system  for  DMA. 

7.2  SOC  2  HIGH  ORDER  LANGUAGE 

DEFINITION:  A  language  in  which  each  instruction  or  statement 
corresponds  to  several  machine  code  instructions;  allowing 
users  to  write  in  a  notation  with  which  they  are  familiar, 
independent  of  hardware.  The  primary  examples  under 
consideration  are  FORTRAN,  COBOL,  and  Ada. 

SON  ITEMS  SUPPORTED: 18,41,57 

A  high  order  language  is  a  software  development  tool 
supporting  the  programming  phase.  High  order  languages  are 
used  because  they  express  a  procedure  and  the  data  being 
manipulated  in  a  fonat  closer  to  common  language  and 
mathematics  than  would  have  to  be  used  if  assembly  or  machine 
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code  were  utilized.  This  format  allows  a  person  to  aore 
quickly  learn  how  to  use  a  computer  because  the  hardware  is 
not  addressed  in  the  languages.  The  use  of  a  high  order 
language  hosted  on  multiple  architectures  provides  a 
communications  interface  for  expressing  problems  between 
different  organizations  working  with  different  machines  or 
applications. 

7-3  SOC  3  SINGLE  LARGE  MULTI-USER  ENVIRONMENTS 

DEFINITION:  Uniform  single  system  tool  bearing  hosts  with 
remote  job  entry  (RJE)  stations  and  the  capability  to  support 
multiple  varieties  and  a  large  number  of  interactive 
terminals. 

SON  ITEMS  SUPPORTED:9,11,<i1,55,60 

A  single  large  standardized  software  development  environment 
at  each  center  would  help  simplify  the  configuration  control 
problem  that  exists.  Total  automation  of  the  system  could  be 
achieved  and  multiple  copies  and  configurations  of  software 
would  be  easier  to  track  and  manage.  Host  large  systems 
usually  support  advanced  word  processing  capabilities  as  well 
as  mail  functions  and  report  generation  facilities.  These 
capabilities  decrease  the  amount  of  paperwork  generated 
manually  in  intermediate  and  final  form.  A  large  system 
would  also  allow  all  departments  to  use  the  same  programming 
support  environment  which  would  provide  a  common  interface  to 
libraries,  tools,  information  distribution,  etc. 

7.4  SOC  4  STANDARD  SMALL  MULTIPLE  ENVIRONMENTS 

DEFINITION:  Uniform  small,  identical  computer  systems  on 
which  to  perform  software  development  each  supporting 
multiple  interactive  terminals. 

SON  ITEMS  SUPPORTED:3,4,36,41,52,55,60 

A  standard  configuration  of  small  software  development 
environments  would  increase  the  physical  interactive  access 
capabilities  by  distributing  the  support  terminals  by 
functional  responsibility  over  a  wider  area.  This  could  be 
accomplished  with  less  effort  than  distributing  terminals 
from  a  central  site.  Response  time  generally  is  decreased 
with  the  use  of  small  systems,  especially  when  a  large  number 
of  terminals  are  to  be  supported.  An  additional  benefit  of 
this  type  configuration  is  that  when  one  system  is  down  for 
maintenance  or  a  scheduled  priority  job,  other  systems  can 
pick  up  the  work  load.  Most  minicomputer  systems  support 
graphics  packages  which  would  allow  the  user  to  generate 
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program  output  in  his  work  area  for  analysis  before  putting 
the  software  into  production.  The  standardization  of  the 
configurations  would  provide  a  common  interface  for 
communication  of  programs  and  ideas  between  systems, 
functional  areas  and/or  centers,  as  well  as  provide  guidance 
to  future  procurements  with  respect  to  the  hardware/software 
interfaces  reguired.  These  systems  could  interface  to  the 
production  mainframes  as  front  ends  allowing  source  data  to 
be  entered  through  CRT,  tape,  disk,  or  cards,  as  appropriate. 

7.5  SOC  5  CONFIGURATION  CONTROL  SYSTEM 

DEFINITION:  An  automated  system  to  track  and  maintain  a 

history  of  changes  to  a  system  or  program  through  development 
and  maintenance  life  cycles. 

SON  ITEMS  SUPP0RTED:9, 41, 42,57,59 

By  definition  this  type  of  system  would  provide  a  means  of 
maintaining  configuration  control  over  the  software  releases 
produced.  Such  a  system  would  also  provide  information  to  a 
user  assistance  function  accurately  and  automatically  which 
could  then  be  distributed  to  users  in  all  organizations  on 
the  latest  updates  in  software.  This  type  of  software  tool 
could  be  used  as  part  of  a  standardized  phased  development 
system. 

7.6  SOC  6  AUTOMATED  OFFICE 

DEFINITION:  Using  computers  to  perform  as  many  typical  office 
tasks  as  possible. 

SON  ITEMS  SUPPORTED : 1 1 , 34, 56 

An  automated  office  system  usually  includes  interactive 
capabilities  to  send  and  receive  messages,  to  generate 
correspondence  and  documentation  using  word  processors  and  to 
invoke  basic  mathematical  functions.  Such  systems  also 
include  hardware  to  support  multiple  output  formats.  This 
type  of  system  would  decrease  the  amount  of  paperwork 
generated,  provide  for  an  automated  means  of  text  management, 
and  supply  management  with  report  generation  capabilities. 

7.7  SOC  7  PROJECT  MANAGEMENT  SYSTEM 

DEFINITION:  Automated  assistance  in  effective  project 

planning,  scheduling,  monitoring  and  control. 

SON  ITEMS  SUPPORTED: 10, 1 1, 12, 14,40,56 
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Milestone  identification,  manloading  projections  and  schedule 
iapact  analysis  are  all  part  of  project  management  systems. 
They  allow  a  manager  to  control  and  analyze  his  projects 
while  generating  a  history  of  the  activities  as  updates  to 
project  plans  are  encountered.  The  paperwork  associated  with 
the  functions  is  reduced  through  interactive  access  and 
magnetic  storage  of  intermediate  data. 

7.8  SOC  8  COST  ESTIMATING  SYSTEM 

DEFINITION:  A  system  used  to  evaluate  software  costs 
associated  with  a  given  project  by  assessing  the  behavior  of 
the  variables  which  impact  life  cycle  cost  and  investigating 
the  project's  sensitivities  to  parameter  changes. 

SON  ITEMS  SUPPORTED: 12, 14,56 

Two  of  the  inputs  into  a  cost  estimating  system  for  software 
development  are  manloading  and  schedule.  A  model  is  built 
which  can  be  analyzed  by  modifying  the  input  variables  to 
bracket  the  original  values.  Using  these  technigues  a 
manager  can  improve  his  evaluations  of  schedule  impacts  and 
manloading  requirements.  As  a  project  evolves,  the  manager 
can  verify/modify  his  inputs  to  provide  historical  data  for 
future  projections  and  to  update  current  projections.  This 
system  would  be  applied  across  all  organizations  and  specific 
"factors"  developed  for  each  which  would  give  an  indication 
of  the  complexity,  size,  volume,  etc.,  of  the  software 
developed. 

7.9  SOC  9  PROJECT  PATH  ANALYSIS  METHOD 

DEFINITION:  A  method  to  organize  project  components,  monitor 

their  progress  and  display  their  status  graphically. 

SON  ITEMS  SUPPORTED: 10, 12, 14,56 

Milestone  identification,  schedule,  and  manloading 
requirements  are  project  components  which  must  be  analyzed 
and  monitored  during  the  software  development  life  cycle. 
Using  project  path  analysis  methods  such  as  CPM  or  PERT, 
these  components  can  be  graphically  displayed  and  tracked, 
providing  a  high  level  visual  source  of  data  for  management. 

7.10  SOC  10  SOFTWARE  ENGINEERING  PRACTICES  TRAINING 

DEFINITION:  A  training  program  to  establish  standard 

practices  for  software  development  and  effective  utilization 
of  human  and  computer  resources. 


SON 


ITEMS  SUPPORTED:2, 18,47,59 


One  task  in  software  engineering  is  to  identify  a 
standardized  phased  development  process  to  be  utilized  in  a 
specific  environment.  Once  the  process  is  established, 
training  is  reguired  to  provide  the  users  with  the  specifics 
of  the  process  which  vary  from  known  textbook  methods  or  from 
another  environment  with  which  personnel  may  be  familiar. 
This  training  is  part  of  any  comprehensive  training  program 
designed  to  provide  quality  assurance  to  products  being 
produced  or  to  help  integrate  new  employees  into  a 
programming  environment. 


7.11  SOC  11  RAPID  PROTOTYPING 

DEFINITION:  A  methodology  used  to  define  programs  or  describe 
program  attributes  in  a  high  level,  possibly  in  non¬ 
procedural  form,  to  provide  the  capability  of  modeling  an 
environment  for  analysis  or  constructing  a  non-production 
program  from  component  parts. 


SON  ITEMS  SOPPORTED:21,22,57 

The  design  phase  of  the  software  life  cycle  is  the  current 
area  of  interest  in  many  academic  communities;  but  the  main 
area  of  study  is  methodologies,  which  provide  no  interactive 
support  for  current  systems.  To  support  users  interactively, 
these  methodologies  need  to  be  supported  through  software 
development  tools  which  verify  their  use.  Additionally  these 
tools  should  be  able  to  support  simulation  of  the  design 
using  some  form  of  program  design  language. 

7.12  SOC  12  AUTOMATED  TRAINING  PROGRAM 

DEFINITION:  An  interactive,  self-paced,  computer  assisted 

program  fully  contained  and  user  friendly  to  be  used  by  an 
individual  familiar  enough  with  his  environment  to  be  able  to 
identify  entry  points  into  the  system. 

SON  ITEMS  SUPPORTED: 12, 18,47 

An  automated  training  program  which  is  self-paced  allows 
employees  to  learn  at  a  rate  which  is  optimum  for  their 
experience  and  background.  Such  programs  also  have  a  minimal 
effect  on  an  employee’s  job  responsibilities  by  allowing  him 
to  train  as  time  and  work  permit.  This  training  should  not 
include  a  topic  that  all  employees  will  need  information 
about  since  this  would  put  a  continuous  load  on  one  source. 
This  type  of  general  training  would  be  covered  by  the  program 
defined  in  SOC  10.  Automated  training  should  be  used  for 
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advanced  or  specialized  topics  and  should  be  a  part  of  any 
coaprehensi ve  training  prograa. 

7.13  SOC  13  AUTOMATED  REQUIREMENTS  GENERATION 

DEFINITION:  A  software  capability  which  can  create  a 

requirements  data  base  relating  to  the  specification  of  a 
system  which  nay  be  analyzed  for  consistency,  completeness 
and  traceability. 

SON  ITEMS  SUPPORTED: 1,5, 41,57, 59 

An  automated  requirements  generation  capability  could  be 
formalized  to  provide  a  requirements  specification  method  for 
DMA  or  a  capability  could  be  developed  to  conform  to  current 
DMA  practices.  This  capability,  or  tool,  could  then  be  used 
to  specify  and  track  requirements  and  changes  to  requirements 
as  a  project  progresses.  A  formalized  method  would  serve  as 
a  communications  interface  between  organizations  requiring 
and  providing  support  and  could  be  standardized  as  part  of  a 
phased  development  scenario. 

7.14  SOC  14  SOFTWARE  DESIGN  LANGUAGE 

DEFINITION:  A  software  design  methodology  and  associated 

system  which  provides  an  effective  communications  medium  to 
support  the  desiqn  and  documentation  of  complex  software 
applications. 

SON  ITEMS  SUP PORT ED :22, 41, 54, 57, 59 

A  software  design  language  should  include  a  programming 
design  language  and  supporting  software  to  verify  use  and 
provide  documentation.  These  systems  are  utilized  to  allow 
communication  at  a  high  level  (more  English-like)  between 
manager  and  designer,  at  a  low  leve..  (more  HOL-like)  between 
designer  and  programmer,  and  additionally  between  designers. 
This  system  should  be  a  part  of  the  standardized  phased 
development  of  software  supporting  the  design  life  cycle 
phase. 

7.15  SOC  15  STRUCTURED  PROGRAMMING  FACILITY 

DEFINITION:  A  menu  driven  collection  of  software  development 

tools  which  includes  a  text  editor  and  library  maintenance 
utilities  designed  to  reduce  keystrokes  and  the  opportunity 
for  error. 

SON  ITEMS  SUPPORTED: 18, 34,55,57,59 
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A  structured  programing  facility  is  an  interactive  software 
development  tool  which  supports  modern  source  data  entry 
techniques,  automated  text  management,  and  system  utilities 
to  perform  common  functions  such  as  sorting,  searching,  and 
basic  math  functions.  The  system  is  usually  menu  driven 
providing  for  a  short  learning  period  to  effectively  exercise 
basic  applications.  This  type  of  system  could  be  implemented 
in  support  of  the  coding  task  of  a  standardized  phased 
development  system. 

7.16  SOC  16  INTERACTIVE  TEXT  PROCESSING 

DEFINITION:  An  automated,  interactive  system  to  build,  print, 
edit,  store  and  retrieve  textual  data,  possibly  including  the 
ability  to  perform  basic  tabulating  and  arithmetic  functions. 

SON  ITEMS  SUPPORTED:11,34 

Interactive  text  processing,  or  word  processing,  can  be  used 
to  decrease  paperwork  where  typing  support  is  required 
through  the  use  of  one-step  function  keys  which  modify  blocks 
of  text,  by  saving  previous  copies  of  text  which  can  be 
easily  modified  and  by  magnetically  archiving  documentation. 
This  task  is  one  part  of  an  automated  text  management  system. 

7.17  SOC  17  AUTOMATED  DATA  COLLECTION 

DEFINITION:  The  ability  to  interactively  assimilate  and 
maintain  information  in  a  chronologically  dependent  format. 

SON  ITEMS  SUPPORTED : 1 1 ,40,46 

An  automated  means  of  collecting  data  could  be  used  to 
decrease  the  amount  of  paper  generated  by  collecting  the  data 
in  magnetic  storage.  The  system  could  also  be  utilized  in  a 
mode  that  would  keep  historical  records  rather  than 
overwriting  previous  data. 

7.18  SOC  18  INTERACTIVE  SUPPORT  SIMULATION  SOFTWARE 

DEFINITION:  A  related  set  of  software  tools  which  simulate 

the  environment  under  which  an  operational  program  will 
execute  by  representing  certain  features  of  a  physical  or 
abstract  system. 

SON  ITEMS  SUPPORTED: 21, 36,57 

Simulation  software  should  be  a  part  of  the  software 
development  tool  set.  This  software  should  be  able  to 
simulate  the  production  environment  subset  with  which  the 
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software  under  development  would  interface,  providing  a  leans 
of  testing  the  software  in  the  programming  environment.  The 
substantial  graphics  environment  of  DMA  should  be  a  prime 
consideration  in  defining  the  simulation  interfaces  and  tool 
output. 
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7.19  SOC  19  SOFTWARE  TESTING  SYSTEM 


DEFINITION:  A  program  analysis  system  used  to  evaluate 
software  and  demonstrate  fulfillment  of  documented  needs. 
Included  are  automatic  test  generators,  data  base  analyzers, 
dynamic  analyzers,  static  analyzers,  test  managers,  etc. 

SON  ITEMS  SUPPORTED:57,58,59 

A  standardized  phased  development  process  must  include  a 
testing  methodology  as  a  software  tool.  The  methodology 
should  be  automated  and  applied  to  both  internally  developed 
and  customer  supplied  software.  Metrics  should  be  generated, 
either  manually  or  by  the  testing  system,  which  could  be  used 
to  gauge  the  software  delivered.  Additionally  the  tool  could 
be  used  on  old  software  to  identify  areas  of  possible 
optimization,  especially  when  an  old  program  is  tested 
against  new  data. 

7.20  SOC  20  SOFTWARE  STANDARDIZATION 

DEFINITION:  An  aid  for  programmers  in  writing  and  checking 
program  documentation/code  and  managers  for  quality 
assurance. 

SON  ITEMS  SUPPORTED:2,57,59 

Software  standardization  is  a  specification  of  the  model  into 
which  a  program  should  fit.  Part  of  the  specification  is 
quality  assurance  standards  identified  by  an  organization. 
Other  parts  include  documentation  to  be  produced  and  testing 
procedures  to  be  followed.  This  specification  should  be  a 
part  of  any  standardized  phased  development  plan. 

7.21  SOC  21  CHARGEBACK  SYSTEM 

DEFINITION:  A  means  of  keeping  precise  account  of  the 
resources  used  by  a  user  to  create  incentives  for  users  to 
conserve  resources. 

SON  ITEMS  SUPPORT ED :44, 46, 48, 52,56 

This  type  system  would  provide  managers  with  information  that 
could  be  used  in  conjunction  with  error  rate  standards  to 
identify  needs  within  an  organization.  An  additional  benefit 
is  that  accounting  data  would  be  more  detailed  and  less 
likely  to  be  erroneously  reported.  With  an  increase  in 
management  visibility  of  computer  resource  allocations  an 
incentive  should  be  created  to  conserve  utilization.  This 
would  help  decrease  turnaround  time. 
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7.22  SOC  22  STRUCTURED  PROGRAMMING 


DEFINITION:  A  style  of  programming  in  which  the  structure  of 
a  program  is  made  as  clear  as  possible  by  using  three  control 
logic  structures:  sequence,  selection  and  iteration. 

SON  ITEMS  SUPPORTED: 58, 59 

The  use  of  structured  programming  practices  results  in 
programs  that  are  readable  and  easily  modified,  hence  easily 
maintained.  In  the  case  of  existing  non-structured  programs 
the  improved  capability  may  not  be  worth  the  effort  of 
modification.  For  original  programs  that  are  not  real-time 
or  time  critical,  structured  programming  should  be  a  part  of 
a  standardized  phased  development  scenario. 

7.23  SOC  23  USER  ASSISTANCE  FUNCTION 

DEFINITION:  An  organizational  function  to  assist  users  in 
overcoming  and  avoiding  errors,  conduct  error  rate  studies 
and  augment  the  information  flow  to  management  enabling 
quicker  response  to  user  needs. 

SON  ITEMS  SUPPORTED: 18,42,44 

A  user  assistance  function  would  help  integrate  new  employees 
faster  by  identifying,  through  trend  analysis,  problems  they 
would  be  likely  to  encounter  and  inserting  them  into  the 
training  curriculum.  For  the  unusual  problems  encountered 
once  a  person  is  trained,  the  function  would  help  to  speed 
their  resolution.  Part  of  the  function  would  be  to  conduct 
error  rate  studies  to  discover  trends  which  might  be  useful 
to  management  concerning  training,  development,  and 
production. 
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8.0  TOOL  SDRVEY 


Inother  part  of  the  Interactive  Computer  Program  Development 
System  Study  was  the  evaluation  and  selection  of  software 
tools  for  the  DBA  MPE.  The  tool  selection  consisted  of  two 
phases.  Phase  I  entailed  the  gathering  of  data  on  the  sof¬ 
tware  tools  available  in  the  near  future  and  their  ap¬ 
plicability  to  the  DMA  programming  environment.  Phase  II  had 
as  its  main  activity  an  in-depth  evaluation  of  specific  tools 
representing  the  classes  of  tools  covering  the  software  life 
cycle  in  a  simulated  Defense  Mapping  Agency  (DMA)  development 
scenario;  as  well  as  an  analysis  of  factors  specific  to  DMA 
which  could  constrain  the  use  of  tools  or  identify  R&D  ef¬ 
forts  to  enhance  their  capabilities. 

The  following  sections  detail  the  activities  which  were  con¬ 
ducted  during  both  phases  of  the  tool  survey  as  well  as  the 
resulting  conclusions  and  related  documentation. 

The  tool  survey  was  an  integral  part  of  the  task  to  develop  a 
Functional  Description  and  System/Subsystem  Specification  for 
a  modern  programming  environment  for  the  Defense  Mapping 
Agency. 


74 


9.0  phase_i_activities 


The  primary  goal  of  phase  I  was  the  gathering  of  data  about 
software  tools.  A  mlti-step  process  was  followed: 

1.  Literature  search 

2.  Selection  based  on  DBA  environment 

3.  Tool  demonstrations 

4.  Analysis  of  demonstration  results. 

The  results  of  this  method  provided  the  basis  for  the  selec¬ 
tion  of  tools  for  the  in-depth  evaluation  phase  (phase  II) . 

9.1  TOOL  INFORMATION  SOURCES 

The  first  activity  involved  a  literature  search  for  software 
tools  applicable  to  the  Defense  Mapping  Agency  software 
development  environment.  Software  tool  directories  and  in¬ 
puts  from  DMA  and  RADC  personnel  were  the  major  sources  of 
information . 

The  following  tool  information  sources  were  used: 

1)  Tools  Fair  -  5th  International  Conference  on  Software 
Engineering 

2)  American  Institute  of  Aeronautics  and  Astronautics 
(AIAA) /Grumman  Software  Tool  Survey 

3)  National  Bureau  of  Standards  (NBS)  Software  Data  Base 

4)  Digital  Equipment  Corporation  (DEC)  Referral  Catalog 

5)  On-Line-Systems  Catalog 

6)  General  Dynamics  Tools  Directory  for  Embedded 
Systems 

7)  Sperry  Univac  1100  Series  Scientific  Software 

8)  Tutorial:  Automated  Tools  for  Software  Engineering 

9)  Tutorial:  Software  Design  Techniques 

10)  Reifer  Consultants,  Inc.  -  Software  Tool  Directory 

11)  Automated  Tools  for  Software  Engineering  Seminar 

12)  Conversion  Products/Aids  Survey 

13)  RADC 

14)  Defense  Mapping  Agency  Headquarters  (DMAHQ) 

15)  Defense  Mapping  Agency  Aerospace  Center  (DHAAC) 

16)  Defense  Happing  Agency  Hydrographic/Topographic  Center 
(DHAHTC) . 
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9.2  DBA  TOOL  APPLICABILITY 


Applicability  of  tools  to  the  DMA  programming  environment  was 
based  upon  several  characteristics.  These  were  selected  af¬ 
ter  analysis  of  the  software  requirements  of  DMA  through  per¬ 
sonnel  interviews,  a  questionnaire  survey  (Appendix  A)  and 
input  from  the  following  government  related  documents: 

1)  FEDSIM  (Federal  Computer  Performance  Evaluation  and 
Simulation  Center)  Installation  Review  -  DBAHTC  - 
November  1980 

2)  DMA  Operational  Concepts  (1982  -  1990)  -  May  1979 

3)  DMA  Program  Support  Library  (PSL)  Interim  Evaluation 
Report,  IBM/FSD  -  November  1980 

4)  DMAAC/Scientif ic  Computer  Division  -  Software  Life  Cycle 
Standards  -  February  1981 

5)  DMAAC  Organizational  Mission  Functions  -  October  1980 

6)  FEDSIM  Installation  Review  -  DMAAC  -  August  1980 

7)  DMA  Modern  Programming  Environment  (MPE)  -  January  1980 

8)  FEDSIM  Optimization  and  Error  Rate  Studies  -  Feb  1981 

9)  Operational  Improvement  Opportunities  for  ONIVAC  1100/80 

10)  DHAHTC  Organizational  Manual. 

The  characteristics  are  defined  as  follows: 

1)  Portability: 

The  capability  of  a  tool  to  be  easily  rehosted  to  a  new 
architecture.  DMA  uses  many  different  computer  systems  in 
their  production  environment. 

2)  Public  domain: 

Any  tool  developed  under  government  funding.  These  tools 
would  be  available  to  DMA  at  minimal  costs. 

3)  FORTRAN  compatible*: 

FORTRAN  is  the  primary  language  used  by  DMA  in  scientific 
computing.  Tools  chosen  should  be  able  to  analyze  FORTRAN 
code. 

♦COBOL  is  also  to  be  considered,  but  not  during  the  tool 
evaluation  phase  of  the  project,  as  recommended  during  the 
In-Process-Review  (IPR)  held  at  R ADC  on  18-19  August  1981. 

The  tools  which  are  available  or  are  under  development  and 
designed  to  work  with  COBOL  will  be  analyzed  by  GD/DSD  for 
the  near-term  and  far-term  environments. 
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4)  ONIVAC  hosted: 


UNIVAC  is  the  primary  production  hardware  used  and  was, 
therefore,  originally  planned  to  be  used  extensively  in  the 
near-term  MPE.  Any  tool  available  on  this  system  would  not 
require  rehost  effort. 

5)  Maturity: 

Maturity  implies  that  the  tool  has  been  in  use  for  some 
tine.  DMA  would  not  have  many  problems  with  this  type  tool 
in  the  area  of  debugging  development  errors  in  its  code. 

6)  User  friendly: 

This  characteristic  provides  for  a  short  learning  curve 
in  tool  usage.  DMA  would  be  able  to  determine  the  effects  of 
tool  usage  on  their  environment  without  a  long  delay  caused 
by  training  requirements. 

7)  DEC  hosted: 

Hosted  on  DEC  hardware.  Many  of  the  varied  computer  ar¬ 
chitectures  used  by  DMA  are  DEC  products. 

8)  Productivity: 

An  increased  capability  to  provide  materials  or  services. 
DMA's  production  output  requirement  is  expected  to  grow 
rapidly  with  image  processing  enhancements. 

9)  Resource  requirements: 

The  amount  of  computer  resources,  including  labor,  neces¬ 
sary  to  perform  a  task.  DMA,  as  with  any  organization,  has 
limited  resources  with  which  to  accomplish  its  assigned  task. 


77 


9.3  TOOL  DEMONSTRATIONS 


The  month  of  June  1981  was  selected  for  tool 
demonstrations/presentations,  two  weeks  at  each  center. 

9.3.1  Tools  Presented 

Several  vendors  agreed  to  give  presentations/demonstrations 
on  the  following  tools: 

1)  On-Line  Systems  -  OSCAR,  OLIVER 

2)  Univac  -  RPS,  CTS ,  MAPPER,  ASET 

3)  Interactive  Systems,  Inc.  -  IS/1  Workbench  for  the  VAX 

4)  Systems  Engineering  Laboratories  -  SOFTOOL  80 

5)  General  Dynamics  Data  Systems  Division  (Eastern  Center) 

-  STAR  1100 

6)  General  Dynamics  Data  Systems  Division  (Central  Center) 

-  PRICE  S 

7)  Logicon  -  LARE 

8)  Grumman  Aerospace  Corporation  -  SOLID 

9)  Gilbert  Commonwealth  -  CUE 

10)  High  Order  Software  -  FAME. 

GD/DSD  DMA  project  personnel  presented  all  other  tools: 

ATA 

AUTOFLOW 

DAVE 

COMPARATOR 

FORMAT 

FORTRAN  1 77  ANALYZER 

LOGOS 

NODAL 

SCMS 

SDDL 

SFTRAN3 

USER  INTERFACE  FOR  ON  LINE  ASSISTANCE  (UIFOLA) 

UPDATE. 
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9.3.2 


Demonstration  Requirements/Schedule 


The  physical  requirements  of  the  presentations  were  discussed 
and  agreed  upon  at  the  Status  Review  held  in  St.  Louis  at 
DMAIC  on  19-20  May  1981  (see  Figure  9.1).  The  demonstrations 
were  held  in  consecutive  two-week  periods  at  DMAHTC  and  DMAIC 
respectively  (see  Figure  9.2).  The  first  two  weeks  of  June 
were  designated  for  presentations  at  DMAHTC  with  duplicate 
presentations  at  DMAAC  the  following  two  weeks.  Those  tools 
outlined  in  cross-hatching  were  presented  by  their  vendor; 
all  other  tools  were  presented  by  G D/DSD  personnel.  After 
each  presentation  a  survey  form  was  completed  by  the  DMA  at¬ 
tendees  to  evaluate  the  tool  with  respect  to  applicability 
and  appropriateness  to  DMA  needs.  The  number  of  personnel 
responding  by  tool  and  center  is  listed  in  Figure  9.3.  A 
copy  of  this  survey  form  is  included  as  Appendix  B. 

The  differences  in  the  schedule  between  the  two  centers 
developed  from  facilities  and  manpower  arrangements.  At 
DMAHTC  the  conference  room  available  was  small  and  a  select 
group  was  chosen  to  participate  in  all  presentations.  The 
limited  seating  required  multiple  presentations  in  most 
cases  to  allow  for  maximum  dissemination  of  information.  At 
DMAAC  the  conference  room  available  was  much  larger  leading 
to  a  reduction  in  the  number  of  double  presentations. 
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FCB  TWO  WEEK  PERIOD  AT  EACH  CENTER 


o  ATTENDEES  EXPECTED:  5-10 

o  STARTING  TIMES 

MORNING  SESSION:  9:30  a.n. 

AFTERNOON  SESSION:  1:00  p.m. 

o  EXPECTED  TIME  REQUIREMENT:  1-2  HOURS  PER  SESSION 

o  PHYSICAL  REQUIREMENTS 

CONFERENCE  ROOM  WITH  TELEPHONE  LINE 

OVERHEAD  PROJECTOR 

35mm  SLIDE  PROJECTOR 

CRT  AND  MODEM 

PRINTER  (132  column  capability) 

VIDEOPLAYER  (optional) 

Figure  9.1  TOOL  PRESENTATIONS  AND  DEMONSTRATIONS 
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DMAHTC 


DMAAC 


ATA:  5 
COMPARATOR :  5 
NODAL:  6 
MAPPER:  4 
OIFOLA :  3 
DAVE:  3 
PORTRAN  77  ANALYZER:  4 
SOLID:  6 
STAR:  7 
SDDL :  8 
SCMS :  6 
INTERACTIVE  1100:  5 
COE:  3 
LARE:  4 
OLIVER:  7 
OSCAR:  5 
SOFTOOL  80:  3 
IS/1  WORKBENCH:  3 
AUTOFLOW:  6 
OPDATE :  6 
FORMAT:  7 
SFTRAN3 :  7 
LOGOS:  7 
FAKE:  4 
PRICE  S:  9 
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Figure  9.3  RESPONSES  BY  TOOL  FROM  DEMONSTRATION 
9.4  DEMONSTRATION  RESULTS 

Most  of  the  DMA  personnel  involved  in  the  demonstrations  did 
not  have  experience  using  software  tools  in  all  phases  of  the 
life  cycle.  As  a  result,  the  data  gathered  was  most  useful 
as  a  me.'ns  of  determining  their  understanding  of  the 
presentations,  but  was  not  heavily  weighted  in  selecting  the 
tools  to  be  evaluated  in  phase  II. 

9.4.1  Analytical  Method 

The  results  of  the  presentations  were  statistically  analyzed 
through  the  tabulation  of  questions  on  evaluation  sheets  dis¬ 
tributed  by  GD/DSD.  The  data  was  then  reduced  through 
several  steps  to  two  lists  from  DMA  responses:  "most 
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positive”  and  "least  negative”.  The  ranking  of  the  tools  was 
based  upon  the  number  of  survey  responses.  Survey  questions 
used  for  ranking  were  based  upon  a  tool's  applicability  to 
DMA  tasks  and  ease  of  use  in  an  interactive  environment.  The 
first  step  in  this  process  involved  gathering  statistics  from 
the  sheets.  The  responses  were  analyzed  for  general 
responses  to  get  a  perception  of  how  well  the  presentations 
were  understood  and  whether  or  not  the  tools  would  be  accept¬ 
able  in  the  DMA  environment. 

Next,  the  data  was  sorted  by  tool  and  then  by  DMA  center. 
Seven  questions  were  selected  to  be  used  in  generating 
numerical  data  to  support  the  general  responses.  These 
questions  are  annotated  in  Figure  9.4.  The  demonstration 
response  form  is  included  as  Appendix  B.  The  statistics  as¬ 
sociated  with  these  questions  are  displayed  in  Figures  9.5 
and  9.6  for  DMAHTC  and  DM AAC,  respectively.  Since  the  objec¬ 
tive  was  to  develop  a  ranking  of  the  tools  by  positive  and 
negative  associations,  the  questions  had  to  be  assigned  a 
corresponding  connotation. 

For  questions,  Q1  and  Q3  (see  Figure  9.4),  a  rating  of  "high” 
was  considered  positive  and  "low”  negative,  since  tlis  im¬ 
plied  ease  of  use.  Questions  Q5  and  Q6  were  considered  to  be 
positive  with  a  "yes”  answer,  the  implication  being  an  ap¬ 
plication  of  the  tool  could  be  perceived  in  the  DMA 
environment.  An  indication  that  modifications  would  be 
required  to  a  tool  in  order  to  make  it  easier  to  use  were 
considered  as  negative  responses. 

This  rationale  lead  to  using  a  "no"  response  to  questions  Q2 
and  Q4  as  a  "positive"  answer. 

Finally,  question  Q7  was  considered  to  have  a  negative 
implication.  This  decision  was  made  based  upon  knowledge 
derived  from  prior  activities  and  discussions.  While  the 
fact  that  a  tool  performs  similar  functions  to  tools  availa¬ 
ble  at  DMA  is  not  in  itself  negative;  a  consideration  must  be 
given  to  the  fact  that  the  tools  at  DMA  are  not  being 
utilized,  for  numerous  reasons.  The  association  of  a  tool 
being  presented  to  a  tool  "available",  but  not  considered 
"useful",  would  be  considered  negative. 

Questions  which  were  not  responded  to  were  not  considered  in 
the  tabulations.  Also,  answers  of  "medium”  for  questions  Q1 
and  Q3  were  classified  as  neutral  responses  in  the 
tabulations. 

9.4.2  Summary  Data 
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The  total  number  of  responses  by  question  and  center  are 
given  in  Figure  9.7,  along  with  summary  data.  Again,  the 
lover  number  at  DMAHTC  can  be  attributed  to  the  approach 
taken  in  participating  in  the  presentations  by  the  center  and 
does  not  reflect  on  the  quality  of  the  statistics.  The  trend 
responses  give  an  indication  of  the  acceptance  to  possible 
changes  in  the  software  tools  available.  A  positive  attitude 
on  the  part  of  the  participants  is  evident,  though  not 
conclusive. 

A  summary  of  all  of  the  statistics  is  given  in  Figure  9.8. 
This  summary  breaks-out  the  data  by  tool,  character  of 
response  and  center  on  the  left  side  of  the  chart.  Numeric 
totals  are  then  collected  in  the  middle  of  the  chart;  first 
by  character  of  response  and  then  without  consideration  of 
any  class  grouping.  Using  the  last  total  the  data  is  nor¬ 
malized  in  the  three  right-most  columns  by  "positive", 
"negative"  and  "neutral"  connotations. 

9.4.3  Tool  Rankings 

Using  the  positive  and  negative  columns  of  normalized  data 
the  rankings  of  tools  by  DMA  in  Figure  9.9  were  derived.  The 
higher  a  tool  appears  in  the  "most  positive"  list,  the  larger 
the  number  of  positive  comments  received.  The  capabilities 
of  some  demonstrated  tools  may  have  been  new  to  the 
evaluators,  and  hence  they  did  not  immediately  perceive  a  use 
for  that  tool.  Therefore,  in  order  to  provide  a  uniform 
baseline  for  ranking  familiar  and  unfamiliar  tool 
capabilities,  the  DMA  survey  responses  were  also  ranked  by 
the  least  number  of  negative  comments  recorded.  The  middle 
column  lists  in  order  the  tools  based  upon  the  number  of 
negative  survey  responses.  Again,  the  best  tool  is  at  the 
top.  The  lower  a  tool  appears  in  the  list  the  larger  the 
number  of  negative  comments  received. 

A  line  was  drawn  connecting  the  same  tool  in  the  most 
positive  and  least  negative  rankings.  The  more  horizontal 
the  slope  of  this  line  the  higher  the  correlation  between  the 
two  ranking  schemes.  A  line  with  a  very  steep  slope  in¬ 
dicates  an  uncertain  correlation.  Such  tools  were  probably 
not  understood,  or  an  application  was  not  apparent.  The  most 
desirable  tools  are  those  that  are  near  the  top  of  both  lists 
and  have  a  high  correlation  indicated  by  a  nearly  horizontal 
line. 

A  third  list,  "most  positive",  was  also  generated  by 
DSD/Central  Center  project  personnel  for  comparison  with  DMA 
responses  (see  Figure  9.9).  The  tool  ranking  was  based  on 
the  needs  of  the  Defense  Mapping  Agency  as  perceived  by  the 


project  team  members  from  their  experience  on  the  task.  The 
higher  a  tool  appears  in  the  list,  the  more  applicable  to  the 
DMA  environment.  Those  tools  that  appeared  at  the  top  of  the 
three  lists  and  had  consistent  correlations  were  desirable 
candidate  tools  for  the  phase  II  evaluation.  As  explained  in 
Section  9.0,  this  was  only  one  of  several  criteria  used  to 
choose  the  phase  II  tools. 
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(GENERAL  EXPLANATION  OF  TERNS) 


QUESTIONS  USED  TO  GATHER  STATISTICS* 

Q 1 ;  Your  evaluation  of  the  ease  of  input  data  preparation: 

_ High, _ Medium , _ Low 

Q2:**  Are  there  modifications  to  the  input  data  preparation 
that  would  make  the  tool  easier  to  use  in  the  DMA 
environment?  _ Yes, _ No 

Q3:  Your  evaluation  of  the  ease  of  understanding  the  output 

results: _ High, _ Medium, _ Low 

Q4:**  Are  there  modifications  to  the  output  results  format 
that  would  make  the  tool  more  useful  in  the  DMA 
environment?  _ Yes, _ No 

Q5:  Do  you  perceive  an  application  of  the  tool  to  DMA 

projectsin  the  near-term  (FY  1982)?  _  Yes,  _  No 

Q6:  Do  you  perceive  an  application  of  this  tool  to  DMA 

projects  in  the  far-term  (FY  1985)?  _  Yes,  _  No 

Q7:**  Does  this  tool  have  functions  that  are  also  present  in 
currently  available  DMA  tools? _ Yes, _ No 


H  -  High 
M  -  Medium 
L  -  Low 

Y  -  Yes 
N  -  NO 

♦Not  all  questions  were  answered  on  all  questionnaires. 
**Negative  Implications 


Figure  9.4  DSD/DMA  TOOL  DEMONSTRATION 
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Figure  9.5  Tool  Demonstration  Responses  -  DMiHTC 


Figure  9.6  Tool  Demonstration  Responses  -  DMAAC 
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Figure  9.8  Tool  Demonstration  Evaluation  Summary 
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10.0  PHASE  II  ACTIVITIES 


In  this  phase  of  the  project,  an  in-depth  analysis  of  the  DMA 
software  development  environment  was  conducted  through  the 
simulation  of  an  actual  development  scenario  using  selected 
tools.  The  tools  being  evaluated  were  representative  of  a 
larger  class  of  software  tools.  The  availability  of 
training,  technical  support  and  computer  resources  were  the 
underlying  considerations  in  the  selection  process.  The  data 
from  the  demonstrations  was  also  used  as  a  selection 
mechanism.  These  constraints  provided  for  an  in-depth,  in¬ 
formative  evaluation  phase. 

10.1  EVALUATION  ACTIVITY  AND  TOOL  SELECTION 

Figure  10.1  summarizes  the  rationale  behind  the  selection  of 
the  tools  for  the  evaluation  activity.  These  tools  were  to 
be  considered  representative  in  nature  of  all  the  tools  under 
consideration  for  the  DMA  MPE.  Data  was  gathered  about  how 
the  tool  could  be  applied  at  DMA,  the  shortcomings  of  its 
use,  the  characteristics  which  seemed  most  productive,  and 
methods  and  training  reguired  to  introduce  the  tool  into  the 
programming  environment.  These  data  were  applied  as  con¬ 
straints  against  all  tools  considered  in  a  specific  life  cy¬ 
cle  activity.  The  implementation  of  this  task  is  described 
in  Section  11.0.  Figure  10.2  presents  the  tools  which  were 
demonstrated  in  June,  1981  and  not  selected  for  the 
evaluation  phase.  The  rationale  of  the  selection/non¬ 
selection  process  is  detailed  in  Sections  10.1.1  and  10.1.2 
respectively. 

An  optimum  evaluation  tool  set  was  chosen  on  the  basis  of 
available  funding,  potential  benefit,  near-term  applicability 
to  DMA,  vendor  support,  and  available  computer/manpower 
resources.  In  addition,  tools  had  to  be  selected  which  would 
cover  all  the  phases  of  the  software  development  life  cycle 
as  well  as  support  associated  management  activities  as 
defined  in  the  SON.  Descriptions  of  the  tools  selected  for 
the  tool  evaluation  have  been  included  as  Appendix  C.  Due  to 
evaluation  constraints,  as  described  in  Section  10.1.5,  only 
two  development  scenarios  were  followed,  one  at  each  center. 
The  development  scenario  was  unique  to  each  center;  however, 
the  test-bed  problem  was  the  same.  Therefore,  in  addition  to 
evaluating  the  individual  tool  capabilities,  it  was  possible 
to  evaluate  the  different  development  scenarios.  GD/DSD  was 
to  develop  the  program  independently  as  a  third  scenario  for 
comparison  with  the  centers.  This  was  only  accomplished 
through  the  design  phase.  Independent  development  by  GD/DSD 
was  stopped  when  it  was  assessed  that  both  centers  would 
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finish  all  life  cycle  tasks  for  the  test-bed  problem  within 
the  specified  schedule. 


Commonalities  between  the  two  center's  planned  scenarios  in¬ 
cluded  the  same  requirements,  testing  and  project  management 
tools;  Front-end  Analysis  and  Modeling  Environment  (FAME), 
Software  Complexity  Measurement  System  (SCMS) ,  OPTIMA  and 
MAPPER  respectively.  Each  center  developed  the  requirements 
for  the  system  using  the  FAME  tool  independently,  and  those 
requirements  were  used  as  the  input  into  the  design  phase. 
The  design  at  DMAHTC  was  accomplished  through  the  expanded 
use  of  FAME  while  at  DMAAC  SDDL  was  utilized.  DMAHTC  used  an 
integrated  software  system  for  coding  and  source 
documentation.  Interactive  Systems/One  (IS/1)  Workbench, 
while  DMAAC  used  TX.  FORMAT  was  to  be  evaluated  at  DMAAC, 
but  access  time  was  limited  due  to  hardware  problems;  and 
only  a  cursory  review  was  accomplished.  NODAL  was  used  as  a 
testing  tool  for  both  centers'  programs.  At  DMAAC  team  mem¬ 
bers  were  shown  at  the  Harris  terminal  how  to  set  up  a  NODAL 
run  for  their  program  files  and  output  from  a  sample  run.  At 
DMAHTC,  due  to  line  problems,  a  live  demonstration  could  not 
be  given.  Instead,  sample  jobstreams  and  outputs  were  shown, 
explained  and  discussed  for  a  small  sample  problem  with  which 
they  were  familiar.  G D/DSD  later  ran  the  DMAHTC  source 
through  NODAL.  SCMS  was  to  be  used  but  the  source  code  was 
not  obtained  from  the  supporting  government  agency  within  the 
evaluation  period.  It  was  preferred  because  of  metrics 
provided  as  part  of  its  output.  By  using  the  same  tool  to 
evaluate  the  testing  phase  of  both  development  scenarios  a 
better  evaluation  of  the  design  and  coding  phase  tools  was 
achieved.  This  did  not  apply  to  the  requirements  phase  since 
both  scenarios  used  the  same  tool.  A  Digital  Land  Mass 
System  was  implemented  in  both  scenarios,  though  the  tech¬ 
niques  and  methods  were  different.  The  coding  at  each  center 
was  -'ccomplished  in  FORTRAN '  66  (FORTRAN  IV).  This  level  of 
FORTRAN  was  dictated  by  the  availability  of  mature  testing 
tools. 

The  project  management  system,  OPTIMA,  was  not  utilized. 
OPTIMA,  a  software  program  developed  and  hosted  on  a  DNIVAC, 
was  not  available  in  an  exercisable  format.  However,  an  in¬ 
teractive  presentation  was  available  on  the  UNIVAC 
Demonstration  and  Presentation  Computer  System  (DAPS)  in 
Eagan,  Minn,  through  a  remote  dial-up  link.  An  additional 
management  tool  exercised  was  a  data  base  inquiry/report 
generator  system,  MAPPER,  used  in  collecting  data  during  the 
evaluation  as  described  in  Section  12.  Access  was  accom¬ 
plished  through  terminals  and  modems  supplied  by  ONIVAC  to 
their  DAPS. 
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10.1.1  Selected  Tools 


The  requirements  tool,  FAME,  was  used  to  specify  the 
requirements  of  the  test-bed  program.  This  tool  was  selected 
after  discussions  with  Higher  Order  Software  (HOS) ,  FAME'S 
vendor.  HOS  offered  more  support  and  better  computer 
resource  utilization  rates  than  other  vendors.  This  tool  was 
also  used  to  specify  the  design  at  DMAHTC.  SDDL  (Software 
Design  and  Documentation  Language),  developed  at  the  Jet 
Propulsion  Laboratories,  was  used  for  the  design  phase  at 
DMAAC.  GD/DSD  had  experience  with  this  tool  and  found  it  to 
be  user-friendly  and  very  productive.  The  computer  resource 
costs  associated  with  the  tool  are  low  because  it  is  hosted 
on  a  minicomputer  and  it  is  highly  portable,  having  been 
written  in  Jenson  and  Mirth  PASCAL.  At  DMAHTC  a  UNIX  based 
system  called  "IS/1  Workbench  for  the  VAX"  was  utilized 
during  the  coding  and  documentation  phases.  This  system  was 
selected  for  use  at  DMAHTC  because  of  the  availability  of 
support.  Interactive  Systems,  Inc.,  the  vendor,  has  a 
Washington  D.C.  office.  A  high  rating  during  the  June  demon¬ 
strations  and  good  vendor  support  in  providing  documentation, 
terminals  and  training,  were  other  reasons  this  integrated 
system  was  selected.  FORMAT  and  TX  were  to  be  used  for  the 
documentation  and  coding,  respectively,  at  DMAAC.  Both  are 
hosted  on  the  same  system  as  SDDL,  a  low  cost  system  with 
good  vendor  support.  Unfortunately,  as  previously  stated, 
FORMAT  was  not  thoroughly  evaluated.  The  testing  tool  used 
by  GD/DSD  was  NODAL.  This  tool  had  fair  ratings  in  the 
demonstrations  and  is  in  the  public  domain.  This  tool  was 
hosted  at  DSD/Central  Center  on  the  SES  system  providing  for 
low  resour  re  costs.  MAPPER  received  the  highest  ratings 
during  th  presentations.  The  UNIVAC  support  was  very  good 
and  there  was  historical  data  on  MAPPER'S  use  from  multiple 
companies.  The  resource  costs  associated  with  this  system 
were  much  lower  than  those  of  other  systems  available  during 
the  tool  evaluation  period.  OPTIMA  was  chosen  as  the  project 
management  tool  for  similar  reasons.  It  was  already  hosted 
on  UNIVAC  equipment  and  could  be  used  at  low  cost.  This  set 
of  tools  kept  the  number  of  vendor  contacts,  computer  systems 
and  training  trips  required  for  the  evaluation  tasx  to  a 
minimum,  while  fulfilling  the  requirement  of  effectively 
evaluating  the  DMA  programming  environment  and  of  exercising 
life  cycle  support  tools.  Figure  10.1  provides  a  summary  of 
this  information. 

10.1.2  Non-Selected  Tools 

The  main  factor  in  not  selecting  tools  was  their  inability  to 
fit  into  the  limited  scenarios  of  the  evaluation.  The  fewer 
hardware/software  systems  which  could  be  used  the  less  com- 
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plex  the  plan.  If  a  tool  was  not  considered  a  high  need  or 
was  not  easily  accessed,  it  was  not  considered  in  the  plan, 
unless  it  was  required  to  cover  part  of  the  life  cycle 
development.  The  only  tool  falling  into  this  category  was 

FAME. 
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10-1.3  Test-Bed  Program 


The  sample  computer  program  to  be  developed  at  each  center 
was  drawn  from  the  cartographic  application  area  and  was 
modeled  after  the  Digital  Land  Mass  System  (DIMS)  problem 
used  in  a  software  engineering  training  class  conducted  at 
DMAAC.  The  problem  was  to  design  a  new  production  program 
which  would  manipulate  a  cartographic  data  base  in  digital 
form.  The  data  base  consisted  of  one  file  containing  four 
manuscripts,  which  could  possibly  overlap.  Each  manuscript 
contained  point,  linear  and  areal  features  in  coordinate  form 
along  with  associated  descriptive  information.  The  program 
requirements  included  the  extraction  of  data  and  features, 
combination  and  merging  of  manuscripts,  updating  the  features 
of  a  manuscript,  and  verification  of  interdependencies  of 
feature  types.  The  number  of  features  per  manuscript  was 
minimized  to  the  extent  that  all  tasks  might  be  accomplished. 
The  entire  program  was  expected  to  be  between  200  and  400 
lines  of  FORTRAN' 66  code.  This  was  an  estimate  and  was  not 
to  be  considered  a  goal.  If  the  time  constraint  did  not  al¬ 
low  for  the  development  of  all  the  requirements,  the  tasks 
were  to  be  narrowed  in  scope  to  allow  completion  of  all  life 
cycle  phases.  A  maintenance  update  activity  would  also  have 
been  undertaken,  however,  at  both  centers  the  scope  had  to  be 
narrowed  because  of  time  limitations. 

10.1.4  DMAAC  Scenario 

In  the  following  two  paragraphs  an  explanation  of  the 
scenarios  with  respect  to  the  support  tools  and  their  inter¬ 
faces  is  provided.  The  first  paragraph  provides  insight  into 
the  original  plan  while  the  second  provides  information  as¬ 
sociated  with  the  problems  encountered. 

10.1.4.1  planned  Scenario 

The  DMAAC  planned  scenario  involved  the  use  of  a  non- 
integrated  set  of  tools  to  define  requirements,  design,  code, 
test  and  document  the  test-bed  program.  FAME,  hosted  on  a 
VAX  and  accessed  over  a  300  or  1200  baud  line,  was  to  be  used 
for  the  test-bed  program  requirements  specification  and  SDDL 
to  design  the  software.  SDDL  access  was  specified  to  be 
through  a  modem  (300  baud)  to  the  Software  Engineering  System 
(SES)  at  DSD/Central  Center  over  a  Harris  terminal  supplied 
by  GD/DSD.  The  Harris  (SES)  full-screen  editor  (TX)  would  be 
used  for  code  entry  and  the  Harris  text  processor  (FORMAT) 
for  documentation  using  the  same  terminal.  SCHS  also  hosted 
on  the  SES  was  to  be  used  for  program  testing,  and  MAPPER  and 
OPTIMA,  hosted  on  UNIVAC  equipment,  was  to  be  used  for  the 
data  base  management  and  project  management,  respectively. 
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Hence,  all  life  cycle  phases  of  the  test-bed  program  and 
automated  project  nanagenent  were  to  be  simulated  in  the 
DMAAC  scenario. 

10.1.4.2  Actual  Scenario 

The  DMAAC  scenario  went  as  planned  with  the  following 
exceptions.  First,  there  were  power  supply  problems  with  the 
Harris  terminal  and  it  was  28  October  1981  before  the  source 
of  the  problem  was  discovered  and  a  replacement  terminal 
delivered.  As  a  result,  a  TYMSHARE  126  and  Harris'  line 
editor  were  used  for  the  design  phase  and  the  start  of  the 
coding  phase.  Also,  as  the  team  was  already  familiar  with 
the  UNIVAC  ED  processor  available  on  their  Dnivac  system, 
FORMAT  was  demonstrated  but  not  used.  In  addition,  the 
DNIVAC  UTS  400  terminal  to  be  used  with  OPTIMA  and  MAPPER  was 
not  received  by  the  team  until  10  November  1981.  As  ex¬ 
plained  in  Section  10.1,  OPTIMA  was  not  available  in  exer¬ 
cisable  form.  Instead,  an  interactive  presentation  was 
available  through  the  UNIVAC  DAPS.  This  meant  that  no  real 
evaluation  of  this  project  management  system  was 
accomplished.  MAPPER  was  available  in  exercisable  form  and 
used  by  individual  team  members  for  storing  their  evaluation 
statistics  once  the  terminals  arrived.  Finally,  due  to  time 
constraints  and  implementation  problems  in  the  hosting  of 
SCHS  on  the  Harris  500  (SES) ,  NODAL  was  used  instead  of  SCMS 
during  the  testing  phase. 

10.1.5  DMAHTC  Scenario 

In  the  following  two  paragraphs,  an  explanation  of  the 
scenarios  with  respect  to  the  support  tools  and  their  inter¬ 
faces  is  provided.  The  first  paragraph  provides  insight  into 
the  original  plan  while  the  second  provides  information  as¬ 
sociated  with  the  problems  encountered. 

10.1.5.1  Planned  Scenario 

The  plahned  DMAHTC  scenario  used  IS/1  Workbench  for  VAX,  an 
integrated  system  similar  to  UNIX,  to  code  and  document  the 
test-bed  program.  The  SHELL,  SCCS  and  MAKE  utilities  of  the 
system  were  to  be  used  for  coding,  and  the  special  word¬ 
processing  functions  for  document  generation.  Access  was  to 
be  through  a  1200  baud  modem  and  INtext  terminal  supplied  by 
Interactive,  Inc  to  a  VAX  system  in  Santa  Monica,  CA.  In  the 
scenario,  rAME,  SCMS,  MAPPER  and  OPTIMA  tools  were  to  be  used 
for  requirements  and  design,  testing,  data  base  management 
and  project  management,  respectively,  hosted  on  the  systems 
indicated  in  Section  10.1.4.1. 
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10.1.5.2  Actual  Scenario 


The  DMAHTC  scenario  was  implemented  as  planned  with  the  fol¬ 
lowing  notable  exceptions.  As  at  DMAAC,  an  OPTIMA  tutorial 
was  available  for  study  but  the  tool  was  not  used  for 
aanageaent  of  the  DLMS  project  due  to  a  presentation  only 
system  being  available.  Also,  as  at  DMAAC,  NODAL  instead  of 
SCMS  was  used  during  the  testing  phase  by  GD/DSD.  In 
addition,  the  SHELL  command  language  and  MAKE  utility,  both 
part  of  the  IS/1  Workbench  for  the  VAX,  were  not  used  during 
the  coding  phase  as  planned.  These  tools,  though  they  can  be 
used  with  FORTRAN  programs,  were  written  to  be  used  with  the 
C  language.  SHELL  did  not  recognize  the  VAX  FORTRAN  command, 
so  command  files  could  not  be  set  up  to  do  compiles  and 
related  functions.  Also,  though  told  MAKE  could  be  used  with 
FORTRAN  programs  to  control  files,  it  apparently  takes  some 
special  handling  and  could  not  be  made  to  work  during  the 
period  allotted  to  the  coding  phase.  Instead,  the  LIBRARY 
command  available  on  the  VAX  was  used  to  create  object  module 
libraries  which  worked  quite  well  for  the  DLMS  test-bed 
program.  SCCS,  Source  Code  Control  system,  was  demonstrated 
and  analyzed  using  small  sample  demonstration  programs,  but 
as  time  did  not  allow  a  maintenance  effort,  this  con¬ 
figuration  control  system  was  not  used  for  the  DLMS  program. 
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Figure  10.1  RATIONALE  FOR  SELECTED  TOOLS 
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Figure  10.2  Rationale  for  Non-Selected  Tools 
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10.2  3D/DSD  TOOL  TRAINING 


Prior  to  the  start  of  the  phase  II  on-site  tool  evaluations, 
DSD/Central  Center  personnel  received  training  and  practice 
in  the  use  of  the  tools  selected.  During  24-25  August  1981, 
training  was  received  in  the  use  of  the  IS/1  Workbench  from 
Interactive  Systems,  Inc.  On  14-15  September  1981  training 
was  received  on  FAME  from  HOS.  Then  on  1  October  1981, 
training  was  provided  by  ONIVAC  on  OPTIMA  and  MAPPER. 
Additional  formal  training  was  provided  on  the  IS/1  system  on 
1  October  with  continuous  training  support  being  provided 
through  an  InterActive,  Inc.  supplied  terminal. 

10.3  DMA  TRAINING 

DSD/Central  Center  personnel  provided  training  to  DMA  person¬ 
nel  on  each  tool  to  be  evaluated  prior  to  the  start  of  its 
associated  life  cycle  phase  on  the  DLMS  test-bed  project. 
The  training  consisted  of  an  explanation  and  demonstration  of 
how  to  access  and  use  the  tool,  objectives  of  the  hands-on 
evaluation,  summary  of  the  capabilities  of  the  tool,  benefits 
to  be  derived  from  the  use  of  the  tool,  sample  scenario  to 
follow  during  the  evaluation,  and  an  explanation  of  data  to 
be  collected  by  participating  DMA  personnel. 

10-4  SCHEDULE  OF  ACTIVITIES 

Resource  constraints,  money,  time  and  manpower  were  some  of 
the  factors  considered  in  establishing  the  phase  II  tool 
evaluation  plan.  At  the  August  1981  In-Process- Review  (IPR)  , 
DMA  indicated  an  availability  of  five  technical  and  two 
managerial  personnel  at  a  50*  level-of -effort  over  an  eight 
week  period.  Allowing  for  a  five  week  time  span  after  the 
IPP  *-o  establish  vendor  coordination,  generate  a  software 
development  scenario,  provide  training  for  project  team 
members,  refine  the  Tool  Evaluation  Plan  (TEP) ,  update  the 
Statement  of  Operation  Need  and  System  Operational  Concept 
(SON/SOC) ,  and  have  a  review  in  St.  Louis,  MO,  a  start  date 
of  05  October  1981  was  selected. 

A  schedule  of  activities  and  on-site  support,  as  accomplished 
during  the  seven  week  evaluation  period,  is  presented  in 
Figure  10.3.  The  period  was  shortened  from  eight  weeks  to 
seven  to  provide  for  maximum  continuity  of  tasks  as  well  as 
due  to  holiday  factors  in  the  calendar  period.  On  the  5th  of 
October,  R.  Bond  and  M.  Goode  arrived  at  DMAHTC,  Washington 
D.C.  to  start  the  tool  evaluation  task.  The  first  day  an 
overview  of  the  effort  was  presented,  including  tools  to  be 
utilized,  access  methods,  schedule  of  activities,  and  tasks 
to  bn  accompJ ished  as  described  in  Section  10.3.  Then 
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specific  tasks  were  assigned.  All  personnel  were  to  work  on 
each  life  cycle  task,  but  a  different  person  would  be  as¬ 
signed  the  lead  role  during  each  life  cycle  phase.  All 
documentation  was  delivered  at  this  time  so  that  individuals 
responsible  for  leading  specific  tasks  (each  life  cycle  phase 
was  a  task)  could  become  familiar  with  the  associated  tool. 
The  next  two  days  entailed  performing  the  requirements  task. 
The  last  day  was  used  to  start  the  design  phase.  Proa  13-16 
October,  a  duplicate  scenario  was  conducted  at  DMAAC,  St. 
Louis  by  R.  Bond.  Each  center  was  responsible  for  continuing 
the  efforts  in  the  assigned  tasks  when  GD/DSD  departed. 

Next,  it  was  planned  that  on  the  22nd  of  October,  M.  Goode 
would  arrive  in  Washington  and  R.  Bond  in  St.  Louis  for  two 
days  to  start  the  evaluation  of  the  project  management  tools. 
Up  to  this  point  in  time  these  tasks  were  to  have  been  accom¬ 
plished  manually.  However,  as  explained  in  Sections  10.1.4.2 
and  10.1.5.2,  the  UTS  400  terminals  to  be  used  with  the 
management  tools  OPTIMA  and  MAPPER  were  late  in  arriving. 
The  trips  were  still  made  as  assistance  trips.  At  DMAAC,  the 
design  phase  was  continued  using  the  Software  Design  and 
Documentation  Language  (SDDL) .  At  DMAHTC  the  design  phase 
was  completed  using  the  Front-end  Analysis  and  Modeling 
Environment  (FAME)  tool  and  coding  was  begun  using  the  INed 
editor  which  is  part  of  the  IS/1  Workbench  for  the  VAX.  An 
extra  trip  was  made  to  DMAAC  on  28  October  to  deliver  a 
Harris  SES  terminal  and  initiate  the  coding  phase  with  in¬ 
troduction  of  TX,  the  Harris'  full  screen  editor.  FORMAT, 
Harris'  text  processor,  was  also  demonstrated.  Simultaneous 
trips  to  each  center  were  made  again  on  02-04  November  when 
the  coding  phase  was  continued  and  testing  begun.  During  the 
DMAAC  visit,  the  evaluation  team  went  to  the  local  Sperry- 
Univac  office  where  the  data  base  management  system  MAPPER 
was  demonstrated  and  report  ID's  set-up  for  each  team  member 
to  be  used  in  the  collection  of  evaluation  statistics.  At 
DMAHTC,  MAPPER  was  introduced  and  IS/1's  Source  Code  Control 
System  (SCCS)  was  demonstrated.  Next,  on  09  November  a  trip 
was  made  to  DMAAC  and  on  10  November  to  DMAHTC  to  deliver 
modems  to  be  used  with  Univac  UTS-400  terminals  in  accessing 
MAPPER.  At.  DMAHTC  MAPPER  was  demonstrated  and  reports  set-up 
for  each  evaluation  team  member.  These  reports  were  used  in 
the  same  manner  as  at  DMAAC  for  the  collection  and  storage  of 
evaluation  statistics.  Finally,  on  16-20  November,  R.  Bond 
and  H.  Goode  traveled  to  both  centers  for  two  days  each.  On 
the  first  day  the  use  of  NODAL  was  evaluated.  The  following 
day  the  management  tool,  MAPPER,  was  utilized  to  assimilate 
the  statistics  which  were  gathered  during  the  evaluations, 
and  discussions  were  conducted  between  GD/DSD  and  DMA  team 
members  on  all  aspects  of  the  evaluation  effort.  These  in¬ 
cluded  the  individual  tools  and  their  features,  the  inte- 
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gration  aspects  of  using  the  tools  to  support  the  various 
life  cycle  phases,  and  the  usefullness  of  the  test-bed  pro¬ 
blem  in  learning  about  the  tools  and  their  capabilities  as 
they  applied  to  the  DMA  environment. 

Statistics  gathered  from  the  test-bed  development  project 
show  a  49*  level-of-ef f ort  by  both  centers.  DHAAC  had  six 
personnel  assigned  to  the  task,  and  DHAHTC  seven.  The  cen¬ 
ters  had  25  and  30  working  days  available  during  the 
evaluation  period,  respectively.  DHAAC  expended  588  hours, 
an  average  of  98  per  person;  and  DHAHTC  expended  828  hours, 
an  average  of  119  per  person;  for  a  total  of  1416  hours. 
This  does  not  include  time  involved  in  the  discussions  con¬ 
ducted  during  the  last  week  of  the  evaluation. 

10.5  3D/DSD  OFF-SITE  EVALUATION  ASSISTANCE 

During  those  times  when  GD/DSD  project  personnel  were  not  on 
site  at  a  DBA  center,  any  guestions  about  the  use  of  a  tool, 
the  test-bed  problem,  etc  were  directed  to  GD/DSD  by  several 
means.  First,  telephone  numbers  for  contact  were  provided. 
Secondly,  the  DHAHTC  team,  while  using  the  IS/1  Workbench, 
had  an  interactive  mail  system  (INmail)  available  to  them 
through  which  they  could  communicate  with  GD/DSD  project  team 
members  in  Fort  Worth,  who  also  had  an  IS/1  terminal. 
Thirdly,  for  the  DHAAC  team  while  using  the  Harris  SES  system 
located  in  Fort  Worth,  special  message  files  were  set-up  for 
communication  with  GD/DSD  project  members. 
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10.6  MANPOWER  UTILIZATION  AND  PHYSICAL  REQUIREMENTS 


Figure  10.4  gives  the  general  requirements  for  facilities  and 
support  provided  by  each  organization.  Seven  people  at 
DMAHTC  and  six  at  DMAAC  were  used  at  a  49*  level-of-ef f or t 
for  a  seven  week  period.  One  person  was  managerial  and  the 
remaining  team  members  technical.  The  manager  was  tasked 
with  the  lead  role  in  utilizing  the  project  management  and 
data  base  tools.  He  had  additional  responsibilities  in  the 
area  of  collecting  statistics  pertaining  to  the  quality, 
productivity,  and  ease  of  use  of  the  tools  in  each  life  cycle 
phase.  The  complete  list  of  statistics  is  included  as 
Appendix  F.  The  method  of  assimilating  the  data  was  manual 
during  the  first  five  weeks  of  the  evaluation,  followed  by 
the  use  of  MAPPER  interactively.  Other  managerial  tasks  were 
to  coordinate  organizational  activities  and  act  as  a  center 
interface  or  contact  point  through  which  problems  and  data 
could  be  transferred  to  GD/DSD  personnel  and  vendors.  The 
lead  role  for  the  individual  life  cycle  phase  rotated  among 
the  technical  personnel.  Each  of  the  technical  personnel  was 
assigned  as  lead  in  one  of  the  five  phases  of  development: 

1)  REQUIREMENTS 

2)  DESIGN 

3)  CODING  (IMPLEMENTATION) 

4)  TESTING 

5)  MAINTENANCE  (DOCUMENTATION) 


The  lead  was  responsible  for  being  knowledgeable  in  the  use 
of  the  tool  associated  with  the  assigned  task.  On  the  first 
day  of  the  effort  at  each  center  the  lead  for  each  phase  was 
chosen.  This  person  was  then  given  the  documentation  on  the 
tool  associated  with  his  life  cycle  phase  role.  The  lead 
studied  the  documentation  prior  to  the  time  he  would  act  as 
lead,  direct  the  efforts  of  the  team  during  the  assigned  life 
cycle  phase,  and  serve  as  the  focal  point  for  discussion  of 
the  tool's  capabilities  during  the  project  review  held  during 
the  last  week  of  the  evaluation  period.  Additionally,  each 
technical  person  acted  under  the  direction  of  the  other 
leaders  in  a  team  effort  to  develop  the  test-bed  program. 

The  physical  requirements  of  each  center  included  a  room  to 
support  three  terminals  with  modems  and  a  blackboard  for 
presentations  by  GD/DSD  personnel.  A  printer  (132  columns) 
was  also  required  and  helpful  to  the  participating  center 
personnel.  One  of  the  three  terminals  needed  was  a  teletype 
(TTY)  and  it  was  requested  to  be  supplied  by  DMA  at  each 
site.  RADC  provided  the  documentation  and  tools  to  be  used 
during  the  testing  phase. 
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Figure  10.4  TOOL  EVALUATION  SUPPORT  REQUIREMENTS 
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10.7  VENDOR  EVALUATION  ASSISTANCE 


Each  vendor  provided  technical  support  throughout  phase  II. 
DMA  personnel  were  directed  to  contact  GD/DSD  project  team 
■embers  if  a  problem  occurred.  If  necessary,  GD/DSD  person¬ 
nel  contacted  the  vendor  for  resolution  of  any  problems. 


107 


11.0  TOOL_EV A LUATION_DOCU MENTATION 

Questionnaires  were  generated  by  GD/DSD  with  DBA  and  BA  DC 
support  to  gather  information  about  the  applicability  of  each 
generic  tool's  implementations  to  the  DBA  programming 
environment.  Each  guestionnaire  was  associated  with  a 
specific  life  cycle  development  phase.  At  the  conclusion  of 
the  evaluation  period  at  each  center,  DSD/Central  Center  per¬ 
sonnel  held  discussions  with  the  tool  evaluation  control 
groups.  The  evaluation  forms  filled  out  by  these  groups  were 
used  to  lead  the  discussions.  These  forms  covered  such  im¬ 
portant  aspects  as  ease  of  access,  usefulness,  and  other 
desirable  features  of  the  tools  evaluated.  Copies  of  the 
forms  are  enclosed  as  Appendix  D.  Appendix  E  contains  a  sum¬ 
mary  of  the  survey  responses  for  each  tool  evaluation  form 
including  comments  by  team  participants. 

Following  the  group  discussions,  GD/DSD  personnel  generated  a 
tool  summary  form,  shown  in  Figure  11.1.  This  sheet  clas¬ 
sifies  each  tool  by  life  cycle  phase  and  important  charac¬ 
teristics  derived  during  the  evaluation  phase,  including  the 
evaluation  teams'  comments  and  reactions  to  the  tools,  which 
were  inputs  for  determining  usefulness  to  DMA. 

Additionally,  the  task  was  documentated  through  the  use  of 
MAPPER,  a  UNIVAC  software  tool,  by  gathering  information  on 
productivity  related  activities  such  as  labor  hours  and  com¬ 
puter  hours  expended,  as  well  as  tool  performance.  The 
productivity  statistics  are  included  in  Appendix  F. 
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Figure  11.1 

Tool  Characteristics  Summary  Form 
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TOOL  EVALUATION  CONCLOSIONS 


During  the  testing  phase  of  the  tool  evaluation  it  was  deter¬ 
mined  that  DMAHTC  generated  332  lines  of  FORTRAN  code,  and 
DMAAC  365  lines.  These  figures  do  not  include  comment 
statements.  Neither  source  was  extensively  tested,  and  er¬ 
rors  are  known  to  have  existed.  With  this  taken  into 
consideration,  the  statistics  show  DMAHTC  produced  .40  lines 
of  FORTRAN  per  hour  and  DMAAC  .62;  or  2.5  hours  and  1.6  hours 
per  line  of  FORTRAN  respectively.  However  this  variance  does 
not  strictly  reflect  the  use  of  a  unigue  scenario  at  each 
center,  with  respect  to  design  and  coding  phases.  Part  of 
the  reason  for  this  difference  is  that  the  DMAAC  team  was 
more  familiar  with  the  stated  problem,  FORTRAN,  and  the  use 
of  software  tools. 

Figures  12.1,  12.2  and  12.3  are  compilations  of  evaluation 
survey  responses  and  activity  statistics.  Figures  12.1  and 
12.2  include  a  summarization  of  the  evaluation  activity 
statistics  by  individual  team.  Figure  12.1,  and  for  the 
total  evaluation  effort.  Figure  12.2.  The  activity 
statistics  as  compiled  on  MAPPER  by  each  participant  have 
bean  included  as  Appendix  F.  Figure  12.3  is  a  compilation  of 
positive  and  negative  responses  by  programming  phase  from  the 
life  cycle  questionnaires  over  the  entire  effort,  see 
Appendix  E. 

Note  that  the  number  of  yes  and  no  answers  for  a  given 
question  and  center  may  not  add  up  to  the  number  of  team 
members.  Two  reasons  for  this  are  (1)  not  every  member  an¬ 
swered  every  question  and  (2)  ambiguous  answers  such  as 
"fairly",  "somewhat",  "maybe"  or  "so-so"  were  counted  as  both 
a  "yes"  and  a  "no".  In  addition,  if  a  range  was  specified 
for  an  answer  requiring  a  number,  the  upper  limit  was  used. 
Also,  comments  were  in  some  cases  paraphrased  to  express  main 
content  and  were  prefaced  with  "AC"  or  "HTC"  to  denote  team 
source. 

An  evaluation  of  the  data  indicates  that  the  team  with 
previous  exposure  to  software  tools,  DMAAC,  did  not  find  the 
tools  to  be  as  useful  as  the  team  with  little  knowledge  of 
tools,  DMAHTC.  This  is  an  important  fact  when  considering 
the  type  of  background  the  personnel  using  newly  introduced 
tools  should  have.  A  positive  attitude  about  a  tool's 
usefulness  will  also  be  important  during  the  transition 
phases  as  new  tools  are  introduced.  One  implication  is  to 
introduce  new  tools  by  first  training  less  experienced 
personnel,  eventually  phasing  in  everyone  as  the  tools  use 
becomes  more  widespread.  This  supports  ideas  expressed  by 


DMA  personnel  when  interviews  were  first  conducted  in  March, 
1981. 

Further  analysis  revealed  a  positive  trend  with  respect  to 
exposure  time  to  a  tool.  For  example,  NODAL  had  the  least 
use  at  each  center,  and  an  accompanying  lowest  rating.  At 
DMAHTC  due  to  line  problems  NODAL  was  discussed  and  sample 
output  listings  explained  but  not  used.  At  DMA AC  time  al¬ 
lowed  the  set-up  and  processing  of  1  run  of  their  program 
through  NODAL.  SDDL  and  FAME  were  extensively  used  at  DMAAC 
and  DMAHTC  respectively;  and  these  tools  evidenced  a  more 
positive  response.  This  relationship  was  corroborated  during 
the  group  discussions  which  indicated  a  need  for  more 
training  time  to  learn  a  tool’s  usefulness. 
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Figure  12.1  Activities  Sumnary  by  Center 


13.0  TOOL  EVALUATION  ANALYSIS 

This  section  details  the  specific  characteristics  that  have 
been  identified  as  important  to  a  tool's  applicability  at 
DMA.  In  general  a  tool  should  be  mature,  system  test  should 
not  be  conducted  by  DMA;  it  should  support  multiple  levels  of 
users,  diagnostics  and  access  should  be  flexible  in  use  with 
respect  to  exposure  time;  and  it  should  be  applicable  to  cur¬ 
rent  DMA  practices,  so  as  to  not  introduce  unnecessary  work 
or  generate  unneeded  data.  Additionally  a  tool  should  sup¬ 
port  ANSI  Standard  FORTRAN  and/or  COBOL;  be  utilizable  in  a 
DNIVAC  mainframe  or  large  minicomputer  environment;  and  be 
wholly  supportable  within  one  DMA  center's  environment. 

Further  analysis  of  the  data  from  the  tool  evaluations 
provided  valuable  information  for  the  near-term  and  far-term 
modern  programming  environment  system  recommendations.  The 
data  was  first  used  by  GD/DSD  in  developing  evaluation 
criteria  for  scoring  different  implementations  which  satisfy 
the  same  System  Operational  Concept  as  defined  in  the 
SON/SOC,  CDRL  A002.  These  Concept  Implementation  Evaluation 
(Cl E)  sheets  are  included  as  Appendix  J.  The  evaluation 
criteria  include  the  following  items: 

Interactive  Capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environment  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Simplicity 
Tool  Use 
Training 
Output 

DMA  Applicable 
Understandable 
System  Resources 

Capabilities  Supported 
Allocations  Required 
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Each  criterion  was  evaluated  by  GD/DSD  on  a  scale  from  0  to 
10.  The  leaning  of  the  numeric  rating  for  each  particular 
criterion  was  defined  using  terminology  appropriate  for  that 
topic.  These  numeric  ratings  were  only  one  part  of  a 
statistical  methodology  used  to  generate  tool  ratings. 

Input  was  sought  from  DMA  and  RADC  for  establishing  weighting 
factors  for  the  evaluation  criteria  with  respect  to  perceived 
importance  for  DMA  on  a  scale  of  1  to  3;  3  being  of  highest 
importance.  In  addition,  GD/DSD,  RADC,  DMAHQ,  DMAAC  and 
DHAHTC  personnel  supplied  inputs  to  help  define  a  priority 
weighting  factor  for  each  need  from  the  SON.  These  are  on  a 
scale  of  5-1  with  a  5  indicating  the  greatest  need  and  1  the 
least.  Using  the  first  weighting  factor  is  analogous  to  per¬ 
forming  a  coarse  rating  with  the  second  weighting  acting  as  a 
fine  tuning  factor.  The  particular  score  for  each  criterion 
is  the  product  of  the  numeric  eva. uation  and  the  weighting 
factor.  The  total  score  for  an  implementation  is  the  product 
of  the  sum  of  the  individual  scores  and  the  need's  priority 
weighting  factor.  The  benefit  of  this  procedure  is  that  it 
establishes  a  uniform  method  of  evaluation  for  all 
implementations.  The  total  score  for  an  implementation  may 
be  low  because  the  need  satisfied  was  not  a  high  priority, 
very  little  information  about  the  system  was  available,  or 
the  ratings  assigned  were  low  values.  The  implementations 
achieving  the  highest  scores  formed  the  nucleus  of  the  tools 
considered  for  the  near-term  modern  programming  environment. 
The  following  definitions  apply  to  the  criteria: 


EVALUATION  RANGES  RELATIVE  WEIGHTING  FACTORS 


High-  (H) 

10-8 

3  =  Very  Important 

Medium-  (M)  = 

7-4 

2  =  Moderately  Important 

Low-  (L)  = 

3-1 

1  =  Not  Very  Important 

No  Information 

Available  = 

0 

13.1  CRITERIA  DEFINITIONS 

1.  Interactive  Capability: 

weight  -  3 

H  -  Highly 

.interactive 

M  -  Partially  interactive 
L  -  Little  or  no  interactive  capability 

The  interactive  capability  of  a  system  is  a  measurement  of 
the  amount  of  manual  or  batch  activity  that  must  be  performed 
when  using  a  system  versus  how  much  of  the  activity  may  be 
accomplished  through  a  remote  terminal. 


in 


* 


•* 


2.  Support  Documentation  Sliaht_f_2 

H  -  Very  thorough  and  understandable  documentation 
H  -  Sufficient  documentation 

L  -  Documentation  very  sketchy  (if  exists)  or  hard  to 
interpret 

Support  documentation  must  be  well  organized,  easy  to  inter¬ 
pret  and  thorough  for  a  system  to  be  used  effectively. 

3.  Diagnostics  -  Documentation  weight  =3 

H  -  Very  thorough  and  understandable  documentation 
M  -  Sufficient  documentation 

L  -  Documentation  very  sketchy  (if  exists)  or  hard  to 
interpret 

Diagnostic  documentation  is  very  important  to  the  utility  of 
a  system.  The  documentation  must  not  be  cryptic  nor  require 
extensive  searching  when  accessed.  Additionally,  it  must  be 
thorough  and  not  be  subject  to  interpretation. 

4.  Diagnostics  -  Interactive  Support  weight  =  2 

H  -  Plenty  of  help  available  while  interacting  with  tool 
M  -  Sufficient  help  available  while  interacting  with  tool 
L  -  Little  or  no  help  available  while  interacting  with 
tool 

Interactive  diagnostics  must  support  multiple  levels  of  users 
including  both  the  novice  and  the  experienced  users.  Use  of 
the  diagnostics  should  cause  minimal  interference  with  the 
work  being  processed.  The  material  content  must  have  the 
same  characteristics  as  the  diagnostic  documentation. 

5.  Automated  Procedure  weight  -  2 

H  -  Highly  automated 
M  -  Partially  automated 
L  -  Not  automated 

A  procedure  is  defined  as  a  set  of  activities  to  be  performed 
in  the  accomplishment  of  a  task.  Automation  is  a  measure  of 
the  interaction  required  by  the  user  of  a  procedure  to 
initiate  the  independent  activities. 

6.  Maturity  weig_ht_=_3 

H  -  Established  -  well  tested  through  actual  commercial 
use 

M  -  On  the  market  -  some  commercial  use 
L  -  State-of-the-art  or  newly  developed  (untested)  yet 
unmarketed 


Maturity  is  a  measure  of  the  time  a  system  has  been 
available,  the  aggregate  utilization  a  system  has  received, 
and  the  state-of-the-art  implementated  by  the  system. 


7.  Vendor  Support  weight  =  1 

H  -  Excellent  -  easily  obtained  -  high  guality 
M  -  Sufficient 

L  -  Poor  quality  -  hard  to  get 

Vendor  support  is  a  subjective  measure  of  the  capability  of  a 
vendor  to  provide  assistance,  personal  and  material,  con¬ 
sidering  items  such  as  physical  location,  workload,  and  past 
experience. 

8.  Availability  weight  =  3 

H  -  Easily  and  quickly  obtained 
M  -  Available  -  may  take  a  little  time  to  get 
L  -  Not  available 


Availablity  refers  to  the  chronological  time  required  for  the 
acquisition  and  installation  of  a  system. 

9.  Hardware  Compatible  weight  =  3 

H  -  Currently  hosted  on  DMA  software  development  hardware 
M  -  Written  in  a  portable  language 
L  -  Extensive  rehost  effort  required 

UNIVAC  1100/62  computers  have  been  identified  as  the  current 
software  development  hardware.  Tools  available  on  this  sys¬ 
tem  may  already  exist  on  DMA  equipment  and/or  would  not 
require  a  rehost  effort. 

10.  Environment  Compatible  weight  =  3 

H  -  Applicable  to  current  DMA  environment 
M  -  Applicable  with  modification  in  use 
L  -  Not  applicable  to  current  software 

The  current  DMA  environment  may  be  described  by  the  hardware 
and  software  in  use  and  the  methodology  of  their  application. 

Each  item  must  be  considered  when  evaluating  environmental 
compatibility. 

11.  Government  Access  weight  =  1 

H  -  Public  Domain 
M  -  With  restricted  rights 
L  -  No  Rights  Available 

A  public  domain  system  was  developed  and  delivered  under 
government  contract  and  would  be  available  at  nominal  cost  to 
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other  government  agencies.  Some  systems  were  commercailly 
developed  and  may  be  purchased  by  the  distribution  and  use  of 
object  and  source  code  in  a  limited  rights  contract.  Other 
commercial  systems  only  allow  the  purchase  of  object  code. 

12.  Flexibility  of  Use  -  Hardware  weight  -  2 

H  -  Portable  with  respect  to  DMA  hardware 
M  -  Portable  in  general 
L  -  Not  very  portable  on  DMA  equipment 

A  software  system  may  be  implemented  on  multiple  computer 
systems  belonging  to  DMA;  exist  on  one  system  and  be  con¬ 
sidered  highly  portable  to  other  hosts;  or  not  exist  on  DMA 
equipment  and  possibly  require  a  rewrite  to  rehost. 

13.  Flexibility  of  Use  -  Software  weiqht_=_2 

H  -  Applicable  to  most  DMA  software 
M  -  Applicable  to  some  DMA  software 
L  -  Applicable  to  minimal  DMA  software 


DMA  software  is  comprised  of  FORTRAN  (66  and  77  standards  and 
extensions) ,  COBOL,  and  multiple  assembly  languages.  Some 
software  systems  have  applicability  across  multiple  languages 
or  language  dialects,  but  in  general  will  be  useful  with  a 
specific  language  implementation.  In  relative  standings 
FORTRAN  is  most  utilizied,  then  COBOL. 

14.  Conceptual  Simplicity  -  Tool  Use  waight_=_2 

H  -  Easily  understood/used 
M  -  Onderstandable/usable  with  effort 
L  -  Complex  in  understanding/usability 

Tool  use  simplicity  applies  only  to  software  tools.  A 
primary  consideration  is  user  friendliness.  This  includes 
ease  of  use  and  understanding  as  well  as  interactive  support 
for  users  with  multiple  levels  of  experience. 

15.  Conceptual  Simplicity  -  Training  weight_=_3 

H  -  Easily  taught/learned 

M  -  Teaching/learning  requires  concerted  effort 
L  -  Complex  to  teach/learn  application 

Training  simplicity  must  be  considered  with  respect  to  DMA 
on-site  capabilities,  the  background  of  the  personnel  in¬ 
volved  including  education  and  experience,  and  the  time  to  be 
involved  in  the  instruction  effort. 
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16.  Output  -  DMA  Applicable  weight  ..=  _  3 

H  -  Complies  with  current  requirements 
M  -  Modifiable  to  current  practices  with  some  effort 
L  -  Not  compatible  with  DMA  requirements 

DMA  has  developed  formal  and  informal  procedures  and  stan¬ 
dards  concerning  support  of  the  software  life  cycle.  These 
procedures  have  bean  developed  over  a  long  time  span  and  each 
new  system  must  be  evaluated  with  respect  to  the  impact  it 
would  have. 

17.  Output  -  Understandable  we iqht _=_2 

H  -  Output  self-explanatory/summary  information  supplied 
M  -  Some  explanation  of  output  initially  required/no 
summary 

L  -  Extensive  training  required 

The  output  of  the  systems  must  be  evaluated  for  clarity  and 
usefullness.  Summary  information  provided  is  a 
consideration.  The  inter pretability  of  the  output,  ab¬ 
solutely  and  relatively,  is  also  a  factor  in  clarity  as  well 
as  usefullness.  Any  training  required  to  understand  the  out¬ 
put  and  its  implications  must  additionally  be  evaluated  as 
part  of  a  systems  criteria. 

18.  System  Resources  -  Capabilities  Supported  weight  =  3 

H  -  Supports  large  number  of  user /hardware/software 
interfaces 

M  -  Limited  interface  capabilities 
L  -  Minimal  interface  capability 

The  resources  supported  by  a  system,  terminals,  users, 
tape/disk  drives,  specialized  peripheral  devices,  etc.  are 
important  to  DMA  due  to  the  large  number  of  users  and  the 
multiple  architectural  devices/interfaces  utilized. 

19.  System  Resources  -  Allocations  Required  weiqht_f_3 

H  -  Minimal  memory/cycles/special  equipment  required 
M  -  Limited  impact  on  resources 
L  -  Heavy  resource  utilization 

When  evaluating  a  system  which  will  be  used  concurrently  with 
or  integrated  into  other  system  software,  computer  resource 
allocation  should  be  a  major  consideraton.  Specific  areas  of 
interest  include  combinations  of  memory,  cpu  cycles  and 
specialized  equipment  required. 
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14.0  ALTERNATIVE  ANALYSIS 


The  Alternative  Analysis  was  the  next  step  in  the  contractual 
effort.  The  results  of  the  Tool  Survey  and  the  SON/SOC  are 
combined  and  analyzed  to  formulate  the  Best-Case  model  for 
the  DMA  modern  programming  environment  (MPE) .  This  Best-Case 
MPE  (reference  Section  15.0)  is  an  unconstrained  model;  it 
provides  the  baseline  for  the  formulation  of  the  Near-Term 
and  Far-Term  MPE's.  Constraints  such  as  schedule 
availability,  cost,  hardware  compatibility,  user- 
friendliness,  technical  capability,  etc.  are  applied  to  the 
Best-Case  MPE  to  arrive  at  the  Near-Term  MPE.  The  detailed 
discussion  of  this  analysis  is  contained  in  Section  16.0.  As 
constraints  are  relaxed,  and  new  technologies  become 
available,  the  Far-Term  MPE  is  defined.  Section  17.0 
describes  this  Far-Term  configuration.  In  Section  18.0  can¬ 
didates  for  future  research  and  development  activities  are 
recommended  to  fulfill  DMA  needs  that  are  not  satisfied  by 
available  technology  and  tools.  Finally,  cost  data  for  the 
Near-Term  and  Far-Term  MPE's  is  contained  in  Section  20.0. 
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15.0  BEST-CASE  MPE 


Figure  15.1  presents  the  evolutionary  process  through  which 
the  Best-Case  MPE  and  Near-Term  and  Far-Term  Modern 
Programming  Environments  were  developed.  The  first  step  in 
the  process  involved  developing  Concept  Implementation 
Evaluation  (CIE)  sheets  and  a  CIE  matrix.  This  matrix  was 
developed  to  provide  continuity  with  the  SON/SOC  document. 
It  contains  the  same  data  as  the  SON/SOC  matrix  with  the  ad¬ 
dition  of  an  "I"  where  one  or  more  implementations  of  a 
specific  need-concept  relationship  exists.  Other  implemen¬ 
tations  may  exist  but  were  not  discovered  during  the  course 
of  the  project.  An  "X"  indicates  the  absence  of  a  known 
implementation.  The  CIE  matrix  is  presented  in  Appendix  G. 
As  described  in  Section  3.0,  the  identified  needs  of  DMA  were 
assigned  weighting  factors.  During  the  Tool  Survey  task  of 
tha  contract,  evaluation  criteria  for  system  operational 
concept  implementations  were  established  and  assigned 
weighting  factors  with  respect  to  the  importance  of  the 
criteria  within  the  DMA  environment.  Implementations  of 
specific  concepts  were  then  numerically  rated  against  the 
criteria  by  GD/DSD.  The  relationship  of  multiplying  the 
criteria  weight  times  the  criteria  rating,  summing  the 
ratings,  and  multiplying  the  total  by  the  need  weight,  which 
derived  an  implementation  score,  was  formatted  into  a  CIE 
sheet.  This  process,  the  criteria,  and  the  weighting  factors 
are  explained  in  Section  13  and  the  sheets  for  all  identified 
implementations  are  attached  as  Appendix  J. 
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Figure  15.1 


NEAR-TERM/FAR-TERM  ENVIRONMENT  EVOLUTION 


15.1  GENERAL  METHODOLOGY 


Utilizing  the  scores  of  the  CIE  sheets  as  a  basis  for  coi- 
parison  of  implementations,  four  rankings  were  developed:  by 
tool,  by  need,  by  concept,  by  score.  These  rankings  are 
presented  in  Appendix  H. 

The  ranking  by  tool  provided  insight  into  each  tool’s  ability 
to  resolve  multiple  needs  as  well  as  the  importance  of  the 
needs  satisfied.  By  ranking  the  needs  the  best  implemen¬ 
tation  to  solve  each  need  was  easily  identified.  This  was 
also  the  reason  for  generating  the  concept  ranking,  to  iden¬ 
tify  the  best  implementation  of  each  concept.  Finally,  rank¬ 
ings  by  high  score  and  specific  tool  were  established.  A 
direct  relationship  exists  between  an  implementation's  score 
and  its  productivity  within  the  DMA  environment;  however  this 
is  only  one  of  many  considerations. 

The  best  (highest  scoring)  implementations  for  each  need  and 
each  concept  were  used  to  generate  Best-Case-by-Need  and 
Best-Case-by-Concept  environments  respectively.  These  two 
environments  were  very  similar.  A  generalized  Best-Case  MPE 
was  generated  by  combining  the  need  and  concept  environments 
and  using  information  available  in  the  tool  and  score 
ranking s.  All  three  environments  are  included  in  Appendix  H. 

The  Best-Case  MPE  was  then  evaluated  in  multiple  steps  for 
near-term  considerations.  In  step  one  the  Ada  language,  for 
structured  programming,  the  Ada  Programming  Support 
Environment  (APSE) ,  as  an  integrated  support  development 
system,  and  the  User  Interface  for  On-Line  Assistance 
(UIFOLA)  were  all  deleted  as  being  for  far-term  consideration 
only.  SOFTOOL  80  was  deleted  as  having  minimal  capability 
when  compared  to  cost  and  need  to  be  satisfied,  as  was  CPAT 
and  the  Planlt  Billback  system.  PRICE-S  was  deleted  due  to  a 
time-share  only  availability.  FAVS  (  or  its  commercial 
equivalent  RXVP80)  and  FORTRAN  77  were  included  in  the  near- 
term  due  to  their  current  application  and  their  high  scores. 
The  MPE  administrator  and  toolsmith  functions  were  included 
on  the  basis  of  the  ease  of  implementation,  and  the  potential 
benefits  to  the  implementation  of  the  MPE  as  a  point  source 
of  information  to  the  user  community.  HYPERGRAPHICS  was 
chosen  for  its  flexibility  in  use  as  a  training  tool,  and  its 
relatively  inexpensive  cost. 

15.2  TOOL  SELECTION  EXAMPLE 

In  Appendix  G,  the  CIE  Matrix,  there  is  an  'I'  located  at  the 
intersection  of  concept  113  and  need  #57,  under 
"REQUIREMENTS".  This  identifies  that  a  need  was  expressed  in 
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the  SON/SOC  document  for  a  software  development  tool  to  be 
used  in  automating  the  requirements  generation  process. 
Further,  the  'I*  specifically  implies  that  one  or  more  im¬ 
plementations  of  tools  to  meet  this  need  are  presented  in  the 
CIE  sheets.  Appendix  J.  The  CIE  sheets,  of  which  there  are 
173,  provide  specific  information  about  a  tools  score  with 
respect  to  the  criteria  listed  previously  in  this  section. 
The  sheet  also  serves  as  a  tabulation  form  for  generating  a 
tool's  total  score  for  an  implementation  as  previously 
described  by  incorporating  coarse  and  fine  weighting  factors. 
The  sheets  are  primarily  collated  by  SOC  number,  then  by  need 
number  within  each  SOC.  Multiple  implementations  within  each 
SOC/Need  sequence  have  not  been  collated. 

Turning  to  SOC  13,  Need  57  in  the  CIE  sheets  it  will  be 
discovered  that  five  implementations  have  been  identified: 
FAME,  R  DP  1100,  PSL/PSA,  SRIMP,  LARE;  with  respective  scores 
of  1289.4,  722.4,  1306.2,  961.8  and  1247.4.  RDP  1100  and 
SRIMP  were  eliminated  due  to  their  low  scores  relative  to  the 
other  tools.  By  determining  the  best  tool  {highest  score)  to 
satisfy  each  need,  PSL/PSA  is  selected.  A  similar  evaluation 
for  the  best  tool  to  satisfy  each  concept  will  have  the  same 
result,  which  eliminates  FAME  and  LARE  as  possibilities.  Now 
PSL/PSA  must  be  evaluated  within  the  constraints  of  the  Best- 
case  MPE. 

DSE. IT  was  identified  as  an  implementation  of  the  SOC  11/Need 
57  relationship.  FAME  is  a  subset  of  0SE.IT»  hence  PSL/PSA 
and  USE. IT  have  duplicating  features.  Inspection  of  the  CIE 
BY  SCORE  listing  shows  values  of  3172.2  for  PSL/PSA,  3131.4 
for  FAME  and  2350.0  for  USE. IT.  This  would  imply  PSL/PSA 
would  be  the  chosen  tool,  except  for  the  fact  that  USE. IT  has 
all  the  functional  capabilities  of  FAME,  so  USE. IT  has  a 
functional  value  of  5481.4.  Additional  information  was 
sought  at  this  point  to  verify  the  selection  of  USE. IT. 

DMA  already  had  PSL/PSA,  although  it  was  not  being  widely 
utilized,  hosted  on  UNIVAC  equipment.  However,  the  in¬ 
formation  obtained  from  the  ISDOS  project  at  the  University 
of  Michigan  indicated  the  UNIVAC  version  was  not  well 
supported;  and  that  future  benefits  will  be  introduced  via 
the  VAX  architecture.  USE. IT  was  already  hosted  on  a  VAX,  as 
was  PSL/PSA.  Further  investigation  provided  the  deciding 
factor.  USE. IT  had  an  interactive  user-friendly  front  end 
with  graphics  capabilities.  A  PSL/PSA  front  end  equivalent 
to  the  USE. IT  system  is  under  development  by  the  ISDOS 
project.  However,  it  is  a  student  based  development  and  may 
not  be  available  or  supported  for  an  indefinite  period. 
USE. IT  also  has  the  capability  of  producing  FORTRAN,  PASCAL 
or,  in  the  near  future,  COBOL  code.  PSL/PSA  is  being  ex- 
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tended  to  produce  FORTRAN  code;  but  again  it  is  a  student 
based  effort.  Additionally,  the  Army  is  utilizing  USE. IT  in 
its  Ada  development  efforts.  Through  this  process  USE. IT  was 
selected  to  support  the  SOC  13/Need  57  automated  requirements 
generation  need. 

15.3  TOOL  BEARING  HOST  SELECTION 

The  second  step  involved  identifying  a  tool  bearing  host 
(TBH) .  A  large  selection  of  mini  and  mainframe  computers 
including  most  major  brands  hosting  FORTRAN,  COBOL  and  sup¬ 
porting  software  development  tools  were  considered;  however, 
only  Harris,  SEL,  UNIVAC  and  VAX  equipment  were  found  to 
provide  the  type  and  extent  of  support  required  by  DMA.  A 
major  requirement  of  the  MPE  concept  was  the  availability  of 
programming  support  environment  tools.  Other  considerations 
were  vendor  support,  DoD  RED  efforts,  physical  size,  ar¬ 
chitectural  capabilities,  and  cost.  The  phase  II  evaluation 
effort  described  in  Section  10.0  determined  the  only  viable 
options  were  VAX  and  UNIVAC.  The  Harris  system's  main 
drawback  was  a  24  bit  word  size  which  would  require  a  repack¬ 
ing  mechanism  when  converting  code  for  use  on  16  bit  produc¬ 
tion  units  or  the  36  bit  UNIVAC  systems.  Additionally,  most 
tools  on  this  system  supporting  life  cycle  phases  other  than 
programming  were  not  mature.  The  SEL  hardware's  32  bit  word 
size  and  through-put  capabilities  were  impressive,  but  the 
sane  problem  existed  with  support  software.  Performing  an 
analysis  of  the  minimum  system  configurations  available  in 
the  near-term  concluded  with  a  recommendation  of  a  VAX  as  the 
near-term  tool  bearing  host. 

In  addition  to  the  statistical  data  available  in  the 
Appendices,  the  major  considerations  were  the  quantity  and 
quality  of  software  development  tools  available  on  the 
systems,  and  the  desire  to  remain  state-of-the-art  while 
evolving  a  far-term  environment.  Though  UNIVAC  was  strongly 
considered  as  a  TBH  for  the  MPE  due  to  its  current  use  within 
DMA  as  a  production  and  development  machine,  the  quantity  of 
software  development  tools  and  the  state-of-the-art 
capabilities  of  tools  currently  hosted  on  the  VAX  are  not 
available  on  UNIVAC.  As  examples,  Ada  and  its  support  en¬ 
vironment  is  currently  being  hosted  on  a  VAX.  There  is  no 
known  activity  proposed  for  rehosting  APSE  to  a  UNIVAC.  In 
the  Software  Configuration  Management  seminars  being  con¬ 
ducted  by  the  Data  Processing  Management  Association  (DPMA) 
many  of  the  manual  techniques  recommended  have  already  been 
automated  by  the  Programmer's  Work  Bench  (PHB)  being  marketed 
under  VAX/VMS  control.  In  the  area  of  requirements 
specification  the  four  major  systems  that  are  available  or 
are  under  development;  PSL/PSA,  BEDS,  RSL/REVS  and  USE. IT  all 
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are  hosted  on  VAX  systems.  Only  PSL/PSA  is  currently  on 
UNIVAC  equipment;  but  according  to  the  ISDOS  project  at  the 
University  of  Michigan  it  is  not  as  well  supported  as  the  VAX 
implementation  nor  does  it  have  as  many  capabilities.  The 
NBS  recently  released  the  FORTRAN  77  Analyzer.  NBS  developed 
its  own  user  interface  on  a  VAX.  As  specifically  relates  to 
the  recommended  MPE,  the  capabilities  provided  through  the 
use  of  USE. IT  and  the  IS/1  PWB  are  not  currently  available  on 
Univac  equipment. 

Also,  information  was  gathered  by  meetings  with  DBA 
personnel,  during  recent  contract  extension  activities,  that 
indicates  a  move  within  both  centers  toward  the  use  of  VAX 
machines.  Hithin  the  next  year  DMAHTC  is  to  obtain  at  least 
8  and  DMA AC  7  VAX  systems.  These  will  be  VAX-ll/780's  for 
the  most  part  and  many  will  be  delivered  with  products.  For 
examples,  DMA  is  to  obtain  the  following  VAX  system  based 
products:  CPS  Clustered  Carto,  TES/EMPS,  Terrain 
Edit/Elevation  and  possibly  the  CPS  Clustered  Carto  system. 
DMA  will  eventually  have  to  provide  maintenance  for  these 
systems. 

Additionally,  there  are  benefits  to  be  derived  through  the 
use  of  a  separate  machine  dedicated  to  software  development. 
The  importance  of  production  runs  normally  results  in  a 
secondary  priority  for  development  runs  thus  reducing  their 
chances  for  a  faster  turnaround.  Moving  development  work  off 
the  production  machine  (s)  results  in  (1)  better  response  time 
for  checkout  and  development  runs  and  (2)  less  interference 
with  production  work. 

Aside  from  life  cycle  system  development  support  tools  the 
following  performance  statistics  were  considered.  For 
'throughput'  comparison,  an  article  in  Datamation,  November 
1980,  indicated  the  KOPS  (thousands  of  operations  per  second) 
rating  of  the  VAX-11/780  to  be  higher  than  an  IBM  370/158,  a 
DEC  2050,  or  UNIVAC  1108  or  1100/60  C2  computers.  The 
whetsone  benchmark  comparisons  in  KIPS  (thousands  of  instruc¬ 
tions  per  second)  at  single  and  double  precision  operations 
support  this  data.  The  benchmark  ’ indicated  the  VAX-11/780 
outperformed  the  SEL  32,  IBM  370/155,  DEC  2050  and  various 
V77  computers  in  double  precision  mode,  while  only  being  out¬ 
performed  by  the  DEC  2050  in  single  precision  mode.  Figure 
15.2  illustrates  the  recommended  near-term  MPE  utilizing  a 
VAX  configuration. 

The  majority  of  recommended  tools  were  developed  on  the  VAX- 
11/780.  These  tools  are  upward  compatible  although  downward 
compatibility  is  not  assured. 
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The  recoaaended  tool  set  addresses  the  significant  needs  of 
DMA  as  identified  in  the  SON/SOC  docuaent.  Ten  of  the  thir¬ 
teen  aost  iaportant  needs,  as  deterained  by  DMA,  RADC  and 
GD/DSD,  have  implementations  identified  in  the  Near-Term  MPE 
vhich  will  immediately  enhance  DMA's  capabilities. 
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Near-Ter*  System  Configuration  for  DBA  Modern 
Programing  Environaent 


16.0  NEAR-TERM  DESCRIPTION 


The  near-term  system  was  selected  through  the  previously 
described  process  to  meet  the  immediate  needs  of  DMA.  As 
defined  the  system  has  a  high  probability  of  improved 
productivity.  A  more  detailed  description  of  the  Near-Term 
HPE  is  provided  in  the  Functional  Description  and 
System/Subsystem  Specification  annexes  to  this  report.  The 
specific  VAX  system  configuration  recommended  evolved  through 
information  obtained  internally  within  General  Dynamics,  data 
provided  by  DMA,  consultation  with  DEC  representatives,  and 
consultation  with  a  major  user  of  VAX  systems.  For 
clarification,  'maintenance  functions'  is  defined  as  post 
production  software  development  activity  requiring  work  in 
one  or  more  phases  of  the  life  cycle:  requirements,  design, 
programming,  testing. 

16.1  DEVELOPMENT  ENVIRONMENT 

A  VAX-11/780  will  be  utilized  for  the  entire  software 
development  life  cycle  including  requirements,  design, 
programming,  and  testing,  as  well  as  configuration  control 
and  project  management  activities.  The  specific  con¬ 
figuration  is  described  in  the  System/Subsystem 
Specification,  CDRL  A007. 

All  software  development  is  performed  under  the  control  of 
the  project  management  tool,  VUE.  Upon  receiving  a  job 
request,  the  project  management  tool  is  initiated  for  the  job 
and  at  various  points  in  the  scenarios,  the  project 
management  system  is  updated  to  reflect  pertinent  decisions 
and  actions.  Examples  of  the  inputs  and  outputs  for  the  VUE 
system  are  illustrated  in  Figure  16.1. 
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Figure  16.1  MPE  Project  Management.  Overview 


For  purposes  of  discussion,  scenarios  will  be  considered  for 
the  following  categories  of  software  development: 


1)  maintenance  of  existing  software  which  has  not  been 
upgraded  by  the  Software  Improvement  program  (SIP) 

2)  maintenance  of  existing  software  which  has  been 
SIP  upgraded, 

3)  software  presently  under  development  for  which 
standards  were  not  specified, 

4)  new  software  to  be  developed  by  DMA  for  which  standards 
will  be  specified,  and 

5)  new  software  to  be  developed  by  contractors  for  which 
standards  will  be  specified. 


The  techniques  discussed  are  intended  to  demonstrate  the  ap¬ 
plicability  of  the  recommended  tools  to  the  various 
scenarios.  Specific  usage  methodologies  will  be  developed 
during  the  MPE  system  implementation  as  outlined  in  Section 
19.1. 


The  application  of  MPE  tools  to  the  DMA  environment  is  illus¬ 
trated  in  Figures  16.2  and  16.3, 
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16.1.1  Approaches 


Within  the  defined  scenarios,  one  of  two  basic  tool  ap¬ 
proaches  will  be  utilized.  These  are  described  in  the  fol¬ 
lowing  two  sections. 

16.1.1.1  Automatic  Programming  Approach 

The  first,  referred  to  as  the  ‘'automatic  programming 
approach",  will  make  repeated  use  of  the  subsets  of  the  tool 
flSE.IT  until  performance  criteria  are  achieved.  The  usage  of 
^he  various  subsets  is  as  follows: 

-  the  USE. IT  graphics  editor  is  used  to  enter  program 
structures,  called  control  maps,  to  functionally 
decompose  requirements  and  design  specifications  as 
well  as  changes,  if  any,  which  are  required  as  a 
result  of  performance  testing, 

-  the  Analyzer  verifies  internal  consistency  and  interfaces, 

-  the  RAT  automatically  produces  programs  from  Analyzer 
output , 

-  source  produced  by  the  RAT  is  compiled  and  linked,  and 

-  the  system  is  performance  tested  to  determine 
acceptability. 

Failure  to  pass  performance  testing  results  in  repetition 
of  these  steps  until  criteria  are  satisfied. 

There  appears  to  be  no  practical  limit  to  the  size  of  system 
which  may  be  developed  with  USE. IT.  As  systems  are  developed 
via  USE. IT,  generic  operations  are  developed  and  can  be 
placed  in  a  library  for  use  as  building  blocks  on  subsequent 
systems.  For  this  reason,  detailed  documentation  within  AXES 
statements  is  considered  mandatory. 

16.1.1.2  Conventional  Tools  Approach 

The  second,  referred  to  as  the  "conventional  tools  approach", 
will  make  use  of  the  SDDL,  DMATRAN/IFTRAN/FORTRAN  or  COBOL, 
and  FAVS/RX VP80  or  CAVS  tools  through  the  life  cycle. 
Utilization  of  tools  in  the  "conventional  tools  approach" 
consists  of  repeated  application  of  the  following  procedures 
until  performance  criteria  are  achieved. 
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USE.IT  is  utilized  to  functionally  decompose  the 
requirements  specifications  and  to  check  the  data  flow 
and  interfaces  on  the  resulting  program  model. 

SDDL  is  used  to  originate  the  design  or  make  design 
changes,  if  any,  which  were  mandated  as  a  result  of  per¬ 
formance  testing 

Source  code  (DM ATR AN /I FTR AN/FORTRAN  or  COBOL)  is 
modified  to  reflect  changes  brought  about  by  design 
changes,  performance  changes,  or  FAVS/RXVP80  or  CAVS 
evaluation 

-  FAVS/RXVP80  or  CAVS  are  envoked  for  the  purpose  of 
detecting  syntax  errors,  performing  static  analysis,  and 
performing  execution  analysis 

Performance  testing  is  evaluated  to  establish  the  ac¬ 
ceptability  of  the  system.  Failure  to  pass  performance 
results  in  repeating  the  process. 

16. 1.2  Scenarios 

One  of  these  tool  application  approaches  is  followed  until 
the  preliminary  test  objectives  are  met.  At  this  time,  the 
source  is  transmitted  via  data  link  to  the  target  host  for 
final  testing. 

While  testing  on  the  target  host,  the  project  management  sys¬ 
tem  is  apprised  of  the  test  status.  Upon  successful  com¬ 
pletion  of  final  test  objectives,  job  completion  data  is 
processed  by  the  project  management  system.  This  action 
prevents  the  system  status  from  being  obscured  from  control 
and  insures  a  match  between  production  software  and  the  as¬ 
sociated  documentation.  Target  host  test  objectives  will 
verify  machine  depandent  devices  and  techniques.  Once  final 
testing  is  completed  and  the  system  is  ready  for  production 
status,  on-line  documentation  such  as  requirement  and  design 
documents,  source  code  and  test  data  should  be  updated  and 
placed  under  configuration  control  using  SCCS.  Under  the 
conventional  approach  all  coding  will  be  accomplished  in  ANSI 
X3. 9-1978  FORTRAN  (77)  or  ANSI  X3. 23-1974  COBOL  (74).  The 
code  should  be  structured  using  the  SEQUENCE,  DOUNTIL, 
DOWHILE,  CASE  control  constructs.  For  FORTRAN  programs  the 
DMATRAN  precompiler  would  be  used  to  translate  the  structured 
code  into  ANSI  standard  code  prior  to  final  compilation  and 
test  on  the  target  production  machine.  Under  the  automatic 
programming  approach  ANSI  standard  code  is  produced. 
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16.1.2.1  Existing  Software 


Upon  receiving  a  maintenance  job  request,  the  project 
management  system  is  provided  with  sufficient  information  to 
make  an  entry  for  the  job.  Should  the  job  request  include 
significant  requirements  modifications  or  is  for  a  system 
which  was  developed  since  the  MPE  installation,  "the  USE. IT 
method",  as  described  above  will  be  utilized  to  rewrite  the 
system. 

For  job  requests  requiring  no  major  requirements  changes  and 
representing  systems  developed  prior  to  the  MPE,  the  SIP 
upgrade  status  of  the  system  is  determined  and  action  taken 
as  described  in  the  following  scenarios. 

16.1.2.1.1  Not  SIP  Upgraded 

Job  requests  for  systems  which  have  not  been  enhanced  by  the 
SIP  program  are  analyzed  for  the  level  of  effort  required  to 
bring  them  to  SIP  standard.  This  level  of  effort  is  compared 
to  that  required  to  express  the  system  requirements  and 
generate  the  system  by  application  of  USE. IT.  If  the  effort 
required  to  bring  the  system  to  SIP  standards  matches  or  ex¬ 
ceeds  the  effort  required  to  re-write  with  USE. IT,  the  system 
will  be  redeveloped  by  the  USE. IT  method  as  described  above. 
Otherwise,  project  management  entries  will  be  made  to  reflect 
the  SIP  upgrade  effort  and  the  system  will  be  brought  to  SIP 
standards  with  tools  and  methods  of  the  SIP  program.  Once 
the  SIP  upgrade  has  been  accomplished,  the  system  will  then 
be  updated  by  use  of  MPE  tools  and  methods  as  described  in 
the  description  of  the  "conventional  tools  approach". 

16.1.2.1.2  SIP  Upgraded  Software 

The  SIP  program  is  intended  to  consolidate  into  a  single 
coordinated  program  many  on-going,  related,  DMA  activities. 
One  of  these  activities  is  the  improvement  of  existing  UNIVAC 
software.  Job  reguests  pertaining  to  systems  having  been 
upgraded  by  the  SIP  program  proceed  through  the  previously 
defined  conventional  development  process.  The  SIP  program  is 
intended  to  consolidate  into  a  single  program  many  ongoing, 
related  DMA  activities  including  an  effort  to  improve  exist¬ 
ing  UNIVAC  software.  The  check-out  process  is  initiated  by 
updating  the  project  management  system  to  reflect  usage  of 
SDDL,  FORTRAN  or  COBOL,  FAVS/RXVF  •  or  CAVS,  and  IS/1  before 
proceeding  to  the  check  out  process.  Upon  achieving  test 
objectives,  on-line  accumulation  of  documentation  including 
requirements,  design  documents,  source  code,  and  test  data  is 
accomplished.  The  accumulated  documentation  is  then  placed 
under  configuration  control,  using  SCCS,  the  project 
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management  system  is  notified  of  the  completion  of  the  HPE 
test  objectives,  and  the  job  is  transmitted  to  the  target 
host  for  final  testing  as  described  above. 

16.1.2.2  Software  Under  Development 

As  a  result  of  the  SIP  program,  systems  under  development 
during  the  transition  to  MPE  will  be  required  to  conform  to 
programming  and  documentation  standards.  At  implementation 
time,  software  contractors  will  not  necessarily  have  access 
to  the  MPE  tools.  For  this  reason,  it  is  recommended  that 
software  under  development  by  contractors  during  the  tran¬ 
sition  to  MPE  be  developed  as  contracted.  In  anticipation  of 
this  action  all  contracts  should  include  a  standard  documen¬ 
tation  definition  and  a  requirement  to  furnish  documentation 
on  a  media  readable  by  the  MPE.  Such  documentation  will  be 
placed  under  configuration  control. 

Those  systems  under  development  by  DMA  during  MPE 
implementation,  should  be  classified  by  priority,  size,  com¬ 
plexity  and  level  of  expended  effort.  High  priority  systems 
and  those  on  which  a  high  percentage  of  estimated  effort  has 
been  expended  should  proceed  as  originally  planned.  Care 
should  be  taken  to  insure  that  these  systems  conform  to  stan¬ 
dards  with  the  final  documentation  being  placed  under  con¬ 
figuration  control.  The  remaining  systems  should  be 
processed  by  the  "automatic  programming  approach",  as 
described  above.  The  development  of  these  systems  will 
provide  invaluable  data  in  the  determination  of  the  cost  ef- 
fectivity  of  USE. IT.  Should  there  be  any  systems  for  which 
USE. IT  development  appears  impractical,  they  should  be 
developed  with  the  "conventional  tools  approach"  of  the  MPE, 
as  described  above.  Action  taken  in  this  development  process 
is  described  in  the  scenario  for  SIP  Upgraded  Software. 

16.1.2.3  Future  System  Development 

To  establish  uniform  systems  which  will  be  more  readily  main¬ 
tained  through  their  life  cycle,  every  effort  should  be  made 
to  have  systems  produced  that  meet  rigidly  enforced 
standards.  It  is  understood  that  these  standards  are  curren¬ 
tly  under  development  and  specific  MPE  related  standards  will 
be  incorporated  as  required  during  final  MPE  implementation 
efforts. 

Since  systems  operated  by  DMA  are  developed  internally  to  DMA 
and  externally  by  software  contractors,  the  development  of 
future  systems  is  discussed  in  the  following  two  scenarios. 
Systems  to  be  Developed  by  DMA,  and  Systems  to  be  Developed 
by  Contractors. 
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16.1.2.3.  1 


Systems  Developed  by  DMA 


Systems  developed  internally  to  DMA  should  make  application 
of  USE. IT.  This  utilization  causes  the  accumulation  of  a  li¬ 
brary  of  operations  as  well  as  putting  development  personnel 
in  an  environment  in  which  structured  system  development  is 
enforced. 

The  application  of  USE. IT  proceeds  as  described  above. 
Successful  completion  of  preliminary  test  objectives  results 
in  reporting  of  this  status  to  the  project  management  and  the 
on-line  accumulation  of  documentation  as  described  in  the 
scenario  for  SIP  Upgraded  Software.  If,  for  any  reason,  the 
application  of  USE. IT  is  not  cost-effective  the  development 
will  proceed  as  described  in  the  scenario  for  SIP  Upgraded 
Software  and  final  testing  on  the  targetted  host  is  performed 
prior  to  completion  of  the  job  reguest. 

16.1.2.3.2  Systems  to  be  Developed  by  Contractors 

Systems  which  are  developed  by  contractors  should  be  done  in 
the  same  manner  as  those  developed  internally.  As  a  minimum, 
contractors  should  be  reguired  to  adhere  to  the  programming 
and  documentation  standards  established  for  DMA  with  all 
documentation  placed  on  a  media  readable  by  the  MPE. 
Ideally,  contractors  will  have  access  to  the  MPE  tools  and 
will  be  required  to  follow  the  methodologies  established  for 
DMA. 

16.2  SUPPORT  ENVIRONMENT 

The  MPE  adminstrator  and  toolsmith  functions  will  support  the 
project  management  function  as  well  as  system  management  and 
training;  and  HYPERGRAPHICS,  a  tool  for  building  presen¬ 
tations  and  interactive  lesson  plans  will  be  utilized  for 
training  purposes.  The  selection  of  HYPERGRAPHICS  is  based 
on  its  simplicity  in  use  as  well  as  cost  and  physical 
availability.  The  generation  of  visual  material  to  support  a 
training  document  is  easily  performed  and  maintained.  A 
training  program  for  Ada/APSE,  a  possible  part  of  the  far- 
term  environment,  has  already  been  developed  on  this  system. 
The  system  was  developed  in  a  university  environment  and  is 
not  expensive;  and  the  capability  of  hosting  on  a  microcom¬ 
puter  allows  physical  access  in  almost  any  area. 

The  VAX  computers  and  UNIVAC  production  mainframe  will  need 
to  be  connected  through  a  communications  link  using  a  stan¬ 
dard  protocol  or  over  an  I/O  channel.  To  support  users  in  a 
timely  manner  and  to  provide  adequate  access,  multiple  VAX 
computers  will  be  required.  The  use  of  multiple  systems  en- 


141 


courages  the  placement  of  MPE  systems  within  functionally 
displaced  areas  instead  of  a  centralized  location  thus  al¬ 
lowing  easier  access  to  the  system.  A  recommendation  of 
three  identically  configured  systems,  resident  at  each  center 
and  intra-center  connected  by  DEC  communications  equipment, 
is  considered  sufficient  to  support  current  activities.  In 
the  future  multiple  systems  could  easily  be  added  as 
required.  Experience  within  GD/DSD  was  one  of  the  primary 
information  sources  used  in  deriving  the  recommended  number 
of  VAX  systems.  Using  well  established  software  development 
areas  as  examples  it  was  determined  how  many  terminals  were 
required  to  support  a  given  number  of  programmers.  Next, 
given  the  number  of  programmers  currently  working  within  each 
DMA  center,  the  number  of  interactive  terminals  required 
within  each  center  for  the  near-term  was  calculated. 
Finally,  using  both  VAX  manuals  and  discussion  with  DEC  re¬ 
presentatives  it  was  determined  that  3  systems  would  op¬ 
timally  support  this  number  along  with  the  software  develop¬ 
ment  environment. 

16.3  RELATIONSHIP  OF  MPE  TO  SIP 

Preliminaly  findings  indicate  that  there  is  little  conflict 
between  the  Software  Improvement  Program  (SIP)  and  the  recom¬ 
mended  MPE.  The  only  potential  conflict  noted  thus  far  is 
the  purchase  of  a  configuration  control  system  under  SIP. 
This  could  result  in  a  duplication  of  the  capabilities  to  be 
provided  by  SCCS  within  the  IS/1  PWB  thus  decentralizing  such 
control  over  production  software  systems.  Other  than  this 
the  two  activities  SIP  and  the  MPE  seem  to  complement  each 
other.  For  example,  both  the  SIP  and  MPE  support  the  use  of 
FAVS  and  CAVS  for  programming  development  and  the  use  of  ANSI 
standard  code.  They  also  each  support  the  use  of  life  cycle 
phases,  structured  programming  practices  and  automated  tools 
during  software  development.  The  MPE  will  benefit  from  the 
SIP  program  in  several  ways.  As  examples,  the  development  of 
a  set  of  software  development  standards  by  DMA  under  SIP  is 
supported  by  the  MPE  and  the  preparations  being  made  under 
SIP  for  the  skills  upgrade  will  benefit  the 
training/introduction  efforts  of  the  transition  to  recom¬ 
mended  near  and  far-term  MPE's.  The  SIP  program  will  also 
benefit  from  the  establishment  of  the  MPE.  Documentation  up¬ 
dates  reguired  by  SIP  for  production  program  upgrades  can  be 
put  under  IS/1  for  configuration  control.  A  USE. IT  support 
library  may  be  developed  as  a  subset  of  the  centralized  li¬ 
brary  which  was  established  under  the  SIP  program.  Such  a 
library  may  be  used  to  enhance  the  rapid  prototyping 
capabilities  of  USE. IT  as  described  in  Section  16.4. 

16.4  USE. IT  TOOL  EVALUATION 
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USB.IT  is  a  tool  based  upon  a  methodology  of  successive 
decompositions  of  requirements  and  design  specifications. 
Information  which  is  known  about  the  problem  is  entered  into 
the  solutions*  model.  Work  needed  to  complete  the  model  is 
apparent  since  the  solutions  structure  is  known  to  be  a  tree 
with  one  root  node  and  leaf  nodes  being  primitives  or  well 
defined  operations. 

Each  data  type  has  associated  with  it  a  set  of  non- 
decomposable  primitive  . operations  (primitives)  which 
manipulate  the  data  for  this  type  in  some  consistent  manner. 

For  example, 

(a)  data  type  "integer”  has  primitives  ADD,  SUB,  IMUL,IDIV; 

(b)  data  type  "vector"  has  primitives  rotate  and  vector 
cross-multiply ; 

(c)  data  type  "screen"  has  primitives  erase,  plot,  point,  and 
unplot . 

These  are  already  available  in  the  USB.IT  library.  The  user 
is  able  to  create  a  new  set  of  data  types  whose  definitions 
may  use  previously  defined  types. 

Operations  such  as 

-  DRAW  BOX  which  draws  a  rectangle, 

-  DRAW  HEXAGON  which  draws  a  hexagon, 

-  DRAW  HYPERBOLA  which  draws  a  set  of  hyperbolas,  and 

-  MIRROR  which  given  a  point  in  one  quadrant  will 
generate  the  image  points  in  the  other  three  quadrants 
by  reflection  about  the  x  axis,  y  axis  and  the  origin, 

are  examples  of  library  functions  that  can  be  constructed  to 
satisfy  the  requirements  of  a  particular  user  problem.  Such 
library  functions  may  be  maintained  in  a  support  library  for 
utilization  on  other  applications. 

The  constructs  of  USE. IT  provide  highly  structured  models 
where  data  flow  is  strictly  controlled.  It  is  this  control 
that  allows  the  model  to  be  inspected  for  correctness.  At 
any  point  in  time  the  model  can  be  analyzed  for  completeness 
and  correctness. 

When  the  solution  has  been  decomposed  from  the  root  node  to 
the  leaf  primitives  or  operations,  the  model  is  complete. 

If  the  inputs,  outputs  and  structures  no  longer  produce  er¬ 
rors  in  the  analysis  phase,  the  model  is  correct  and  FORTRAN 
code  can  be  automatically  generated  to  produce  the  solution. 
Evaluation  results  suggest  that  any  problem  can  be  decomposed 
employing  USE. IT.  A  LOHAN  navigational  lattice  problem  was 
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used  for  the  evaluation  model.  A  complete  problem  definition 
can  be  found  in  Appendix  I. 

For  the  LORAN  problem,  23  man  days  were  spent  in  preparation 
for  formal  presentation  of  the  problem  solution.  Of  these, 
17  were  spent  in  development  of  user  level  operations  while  6 
were  used  in  applying  the  operators  to  the  problem  solution. 

Several  findings  concerning  the  USE. IT  tool  have  already  been 
obtained  using  the  LORAN  problem  as  a  test-bed.  First,  the 
USE. IT  tool  forces  structured  development  through  the  use  of 
its  constructs,  constructs  which  are  best  understood  by 
programmers.  The  USE. IT  tool  requires  the  software  developer 
to  concentrate  on  what  is  to  be  accomplished  instead  of 
having  concerns  over  how  the  task  is  to  be  accomplished.  For 
best  use  of  USE. IT,  patterns  of  thought  must  match  the  tool's 
approach  of  decomposition.  Individuals  who  already  produce 
structured  code  should  find  USE. IT  easy  to  use. 

The  FORTRAN  code  produced  by  USE. IT 

(a)  is  error  free, 

(b)  is  not  structured,  although  FORTRAN  77  constructs 

may  be  modeled  with  AXES, 

(c)  contains  very  few  comments, 

(d)  has  variable  names  which  are  nondescriptive, 

(e)  and  is  inefficient  as  compared  to  code  produced  by  an 
experienced  programmer. 

However,  the  maintenance  of  USE. IT  produced  FORTRAN  code 
would  defeat  the  purpose  of  the  tool  since  respecification  is 
the  correct  way  to  maintain  a  model. 

To  reduce  inefficiencies  in  the  FORTRAN  code,  higher  level 
operations  can  be  coded  and  placed  in  a  library.  This  ap¬ 
proach  is  highly  encouraged  since  a  library  of  models  can 
reduce  the  detail  of  specification,  reduce  repeated  produc¬ 
tion  of  the  same  model,  and  reduce  the  number  of  subroutine 
calls. 
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17.0  PAR-TERM  DESCRIPTION 


The  recommended  software  tools  and  hardware  tool  bearing  host 
(TBH)  for  the  Far-Term  Modern  Programming  Environment  are 
shown  in  Figure  17.1.  The  recommendation  of  the  VAX-11/780 
as  the  far-term  tool  bearing  host  is  based  upon  two  important 
facts.  First,  the  VAX-11/780  will  already  be  on-site  and 
available  as  the  TBH  because  it  is  also  recommended  in  the 
near-term.  Consequently,  the  near-term  to  far-term  tran¬ 
sition  will  be  smooth  and  will  not  disrupt  DMA  software 
development  and  maintenance  activities.  Secondly,  the  trends 
in  the  software  industry  are  to  host  many  development  aids 
and  tools  on  a  VAX  system.  Examples  are  the  Ada  language 
compiler  and  the  Ada  Programming  Support  Environment  (APSE)  . 
Therefore,  it  is  anticipated  that  DMA  will  have  ready  access 
to  future  tools  that  will  be  developed  and  hosted  on  the  VAX. 

17. 1  SOFTWARE  TOOLS 

The  recommended  software  tool  suite  supports  all  the  software 
development  life  cycle  phases.  The  USE. IT,  SDDL , 
DMATRAN/IFTRAN,  IS/1  (with  INword,  INed,  and  SCCS) ,  FORTRAN 
77/COBOL  74,  and  FAVS/RXVP80  or  CAVS  tools  support  the 
requirements,  design,  coding,  maintenance  (documentation, 
text  editing,  and  configuration  control) ,  maintenance  coding, 
and  testing  tasks,  respectively.  All  of  these  tools  will  be 
in-house  and  operational  on  the  VAX-11/780  from  the  near-term 
environment . 
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17.1.1  Ada 


Ada,  the  DoD  standard  high  order  language,  is  scheduled  to 
h?ve  a  compiler  hosted  and  targeted  to  the  VAX  by  the  begin¬ 
ning  of  1583.  The  MAPSE  (Minimal  Ada  Programming  Support 
Environment)  consists  of  a  loader,  data  manager,  compiler, 
editor,  linker,  and  command  interpreter  all  integrated  into  a 
smoothly  working  tool  set.  The  APSE  consists  of  the  MAPSE 
plus  user  specific  tools.  The  architecture  of  the  APSE  is 
designed  to  permit  easy  integration  of  user  tools.  The  MAPSE 
on  a  VAX  is  scheduled  for  completion  by  the  end  of  1984  and 
should  be  available  for  the  far-term  environment.  It  is 
recommended  that  these  Ada  capabilities  be  included  in  the 
far-term  environment  so  that  DMA  can  benefit  from  this  new 
technology  and  from  the  Ada  tools  produced  by  industries, 
government,  and  universities. 

17.1.2  Rehost  Efforts 

In  the  far-term  environment  it  is  recommended  that  one  tool 
be  rehosted  from  the  near-term  environment  to  the  VAX.  The 
HYPERGRAPHICS  tool  would  probably  be  an  easily  rehosted  sof¬ 
tware  package.  HYPERGRAPHICS  was  already  recommended  in  the 
near-term  and  would  be  a  proven,  mature  tool.  Also,  as  men¬ 
tioned  previously,  an  Ada/APSE  training  program  was  developed 
on  this  system  that  could  be  used  as  an  aid  in  transitioning 
to  the  far-term  MPE. 

17.1.3  Development  Efforts 

One  effort  involved  in  the  upgrade  from  near-term  to  far-term 
MPE  will  be  the  development  of  logical,  automated  interfaces 
between  the  life  cycle  development  tools.  This  means  that 
the  output  from  one  tool  should  be  automatically  matched  to 
the  input  required  by  the  next  phase's  tool.  This  could  in¬ 
volve  tool  modifications  and/or  the  addition  of  postproces¬ 
sors  or  preprocessors  to  be  used  between  phases.  Essentially 
the  resulting  system  would  appear  to  the  user  as  1  tool  with 
1  interface.  Additionally,  a  centralized  software  develop¬ 
ment  database  might  be  developed  as  a  common  information 
source  and  storage  area  for  use  by  all  life  cycle  development 
tools. 

Also,  at  this  time  a  fully  satisfactory  project  management 
tool  for  the  VAX  has  not  been  identified.  The  particular 
capabilities  of  a  software  cost  estimating  tool  and  a 
chargeback  accounting  system  could  be  obtained  in  one  of  two 
ways.  First,  a  full  capability  project  management  system  for 
the  VAX  may  be  developed  by  a  software  vendor  and  it  may 
satisfy  DMA  needs.  If  so,  it  can  be  acquired  and  used.  The 
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other  option  is  to  contract  the  development  and  installation 
of  a  project  management  and  accounting  system  that  is  par¬ 
ticularly  tailored  to  the  DMA  organization.  There  is  a 
definite  need  for  automated  tools  to  support  project 
management  in  the  far-term  MPE. 

Finally,  methods  should  be  developed  to  communicate  through 
the  software  development  MPE  systems  with  commonly  used 
databases  already  existing  at  the  DMA  centers. 


i 

i 
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17.2  KEY  POINTS 


In  summary,  the  key  points  in  the  Far-Term  Modern  Programming 
Environment  are: 

(1)  Use  of  the  VAX-11/780  as  the  tool  bearing  host 

(2)  Carry  over  of  the  proven  near-term  tools 

(3)  Introduction  of  Ada  to  prepare  for  future  technology 
and  tool  development  if  DMA  elects  to  use  Ada 

(4)  Continued  support  of  the  FORTRAN  and  COBOL  environ¬ 
ments 

(5)  Logical  integration  of  the  life  cycle  development 
tools 

(6)  Common  formatted  software  development  database 

(7)  Ability  to  write  code  for  additional  DMA  production 
machines  besides  UNIVAC  and  VAX. 

This  plan  offers  to  DMA  the  benefits  of  low  cost  for  the  far- 
term  environment  and  the  miviimum  risk  in  technical  and 
schedule  areas. 
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18.0  R&D  ACTIVITY 


This  section  describes  new  software  engineering  tools  to  bet¬ 
ter  satisfy  DMA  needs  as  well  as  modifications  to  existing 
tools.  The  purpose  of  these  research  activities  is  to  fine- 
tune  the  Far-Term  MPE  to  maximize  efficiency  in  the  software 
development  process.  Anticipated  benefits  include  maximizing 
the  productivity  of  programmers,  eliminating  redundancy  among 
projects  by  establishing  libraries  of  reuseable  software, 
providing  a  capability  to  accurately  project  and  track 
development  costs,  and  being  in  a  position  to  reap  the 
benefits  of  other  DoD  research  activities.  Other  potential 
benefits  include  the  identification  of  previously  un¬ 
discovered  needs  at  DMA,  and  the  capability  to  accomplish  yet 
undefined  software  tasks  without  a  major  effect  on  the 
production  environment. 

18.1  NEAR-TERM  MPE  SPECIFIC 

The  Near-Term  MPE  is  a  set  of  tools  with  specific  uses 
recommended.  These  tools  have  potential  utilization  beyond 
the  scope  of  this  study.  These  potentials  should  be  in¬ 
vestigated  for  application  to  the  Far-Term  environment. 

18.1.1  IS/1  System  Implementation 

The  programmers  work  bench  (PWB)  uses  two  main  tools,  SCCS 
and  MAKE.  Both  tools  will  need  default  values  and  implemen¬ 
tation  parameters  specified  with  respect  to  the  DMA  environ¬ 
ment  standards  established.  These  items  invoke  specific 
compilers,  enforce  naming  conventions,  update  release 
versions,  provide  for  limited  access  and  numerous  other  func¬ 
tions  which  must  be  standardized  across  all  systems  as  well 
as  molded  to  specific  DMA  requirements.  A  supporting 
methodology  for  utilization  of  these  tools  must  also  be 
developed . 

18.1.2  USE. IT 

For  reasons  of  compatibility,  the  USE. IT  developer  should  be 
contracted  to  produce  FORTRAN  77  source  code.  It  should  be 
noted  that  USE. IT  produces  FORTRAN  66  code  which  is  a  proper 
subset  of  FORTRAN  77.  The  constructs  available  under  FORTRAN 
77  may  be  developed  under  USE. IT  for  use  in  developing 
program/system  models  utilizing  USE. IT. 

A  USE. IT  operation  library  should  be  developed  and  maintained 
as  a  subset  of  the  program  support  library.  Criteria  for  in¬ 
clusion  in  the  library  should  be  established.  Rapid 
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prototyping  within  the  DMA  HPE  is  dependent  on  the  ac¬ 
cumulation  of  a  set  of  DMA  specific  operations. 


Although  USE. IT  is  the  most  promising  tool  available  to  ac¬ 
complish  the  automatic  programming  approach,  further  research 
is  required  in  the  areas  of  training,  DMA  programmer 
acceptance,  time  critical  systems  performance,  commitment  of 
developer  to  produce  other  source  languages,  and  the  ability 
of  the  developer  to  provide  long-range  support.  Access  to 
USE. IT  by  potential  DMA  users  for  test-bed  experimental 
development  efforts  would  be  very  beneficial  in  researching 
these  areas.  Emphasis  should  be  placed  on  the  interface 
between  the  two  high  level  tools,  USE. IT  and  SDDL.  Although 
preliminary  considerations  indicate  the  feasibility  of  the 
task,  further  research  is  required  to  implement  the 
interface. 

18.1.3  VAX-11/730  Experimental  System 

To  provide  convenient  user  access  and  the  most  cost  effective 
implementation  of  the  experimental  system,  the  ability  to 
host  the  the  HPE  tools  on  the  VAX-11/730  should  be 
considered.  Points  to  be  researched  include:  (a)  ability  to 
host  recommended  HPE  tools  on  the  11/730;  (b)  expected  per¬ 
formance  ;  and  (c)  extent  of  possible  upgrade. 

18.1.4  FAVS/RXVP80 

DMATRAN  and  FAVS  were  created  from  earlier  versions  of  the 
General  Research  Corporation  IFTRAN  and  RXVP80  software  tools 
in  an  effort  to  customize  them  to  fit  the  DMA  environment  and 
UNIVAC  systems.  Since  the  creation  of  DMATRAN  and  FAVS,  the 
commercial  versions  IFTRAN  and  RXVP80  have  undergone  many 
upgrades  and  changes.  Though  these  commercial  versions  are 
available  on  the  VAX,  there  is  some  question  as  to  their  ap¬ 
plicability  to  the  DMA  environment.  The  acceptability  of 
IFTRAN  and  RXVP80  to  satisfy  DMA  requirements  should  be 
researched  before  decisions  can  be  made  whether  to  rehost 
DMATRAN  and  FAVS  or  purchase  their  commercial  counterparts. 

18.1.5  Advancements  in  the  State-of-the  Art 

The  recommended  tools  and  hardware  represent  the  current 
state-of-the-art  at  the  time  of  this  writing.  Due  to  their 
present  technological  position,  it  is  anticipated  that  they 
will  remain  on  the  leading  edge  of  technology.  Obviously, 
should  this  position  change  prior  to  implementation  of  the 
DMA  HPE,  systems  should  be  acquired  that  are  deemed  to  best 
fit  DMA  needs  at  that  time. 
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18.2  FAR-TERM  MPE  SPECIFIC 


The  Far-Term  MPE  is  based  upon  software  development  tools  ex¬ 
pected  to  be  available  by  1987  and  their  relationship  to  the 
tools  and  hardware  recommended  in  the  Near-Term  MPE.  The 
Par-Term  MPE  is  described  through  general  recommendations  of 
the  configuration  with  supporting  general  operational 
concepts. 

18.2.1  Cost  Estimating 

A  software  cost  estimating  capability  needs  to  be  developed 
for  DMA.  This  capability  has  two  aspects:  (1)  a  methodology 
and  (2)  an  automated  tool;  both  of  which  need  to  be  par¬ 
ticularly  adapted  for  DMA  use.  A  research  activity  whose  ob¬ 
jective  is  the  development  of  a  software  cost  estimating 
capability  would  study  in  detail  the  technical  software 
development  and  maintenance  efforts,  the  management  review 
procedures,  and  the  working  environment.  The  benefit  of  the 
research  would  be  improved  management  control  of  costs  and 
schedules. 

18.2.2  Management 

The  management  tools  must  also  form  an  integral  part  of  this 
methodology.  This  would  reguire  modification  of  available 
systems,  or  possibly  their  known  method  of  utilization,  or 
the  development  of  DMA  specific  systems.  The  major  problem 
with  current  tools  is  the  level  of  users  supported.  In  non- 
automated  systems  such  as  PERT  a  user  may  choose  any  or  all 
parts  of  the  methodology  as  applicable.  In  the  automated 
systems  all  parts  must  be  activated  and  few  sections  have 
default  values,  hence  a  large  pre-utilization  effort  is 
required  to  establish  metrics  independent  of  the  level  of 
support  required. 

18.2.3  Tool  Set  Integration 

The  recommended  tool  set  provides  the  state  of  the  art  in 
programmer  aids.  To  further  optimize  programmer  activity,  an 
effort  to  integrate  the  tools  and  their  interfaces  should  be 
undertaken. 

18.2.4  Code  Auditor 

In  order  to  enforce  coding  standards  established  by  DMA,  a 
research  effort  should  be  initiated  for  a  coding  auditor 
system.  The  purpose  of  such  a  system  is  to  isolate  portions 
of  code  which  do  not  comply  with  established  standards. 
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18.2.5  Ada 


The  Ada  language  has  been  developed  to  aid  in  the  deter- 
aination  and  iapleaentation  of  new  system  development 
standards.  It  is  envisioned  that  Ada  will  be  used  throughout 
the  life  cycle  of  all  new  systems  from  requirements 
specification  through  system  maintenance.  The  specific  pro¬ 
blem  is  providing  data  on  the  use  of  Ada  capabilities  as  ap¬ 
plied  to  the  life  cycle  methodology  for  large  scale  systems 
development.  To  implement  a  total  life  cycle  methodology, 
the  following  tasks  must  be  performed.  First,  information 
must  be  collected  on  the  use  of  Ada  in  the  design  of  large 
scale  systems  and  on  the  training/skill  levels  needed  to  do 
this  defined  task.  Next,  this  information  must  be  utilized 
to  create  new  development  standards  and  associated  curricula 
of  instruction  to  train  the  work  force.  Finally,  the  new 
standards  must  be  provided  to  the  work  force  so  that 
utilization  of  the  standards  and  generation  of  the  expertise 
needed  at  the  systems  specification  and  systems  design  levels 
will  be  achieved.  This  problem  is  a  sigificant  part  of  a 
much  larger  problem,  the  high  cost  of 
development/maintenance,  and  its  solution  will  contribute  a 
great  deal  to  the  advancement  of  system  capabilities. 

Additionally,  due  to  the  current  status  of  Ada  and  its  sup¬ 
porting  environment,  the  adaptability  of  Ada  to  the  DMA  en¬ 
vironment  is  not  presently  established.  However,  the  desire 
of  the  Department  of  Defense  to  have  a  standard  language  and 
the  strong  features  of  the  language  indicate  Ada  will  likely 
be  used  by  DMA  at  some  time  in  the  future.  Scenarios  of  Ada 
usage  are  another  topic  for  further  study. 


153 


* 


19.0  CONCLUSIONS  AND  RECOMMENDATIONS 

These  recommended  near-tera  and  far-tera  environaents  meet 
the  requirements  as  specified  in  the  SON/SOC  as  well  as 
provide  for  the  environmental  capabilities  identified  during 
the  software  tool  evaluation.  In  the  Near-Term  HPE  risks 
have  been  minimized  by  recommending  tools  which  are  currently 
available  and  have  been  throughly  investigated  with  respect 
to  claimed  performance  capabilities.  Performance  cannot  be 
quantified,  but  cost  data  and  rationale  are  provided  which 
support  our  conclusions.  An  experimental  system  should  be 
developed  first  in  the  implementation  of  the  environment  to 
provide  engineering  data  to  fine  tune  system  performance. 
Additional  data  derived  from  this  system  will  contribute  to 
the  development  of  tool  usage  methodologies,  standards,  and 
training  programs  during  the  inplementation  of  the  Near-Term 
and  Far-Term  MPE's. 

19.1  TRANSITION  PLAN 

The  process  of  transitioning  to  the  Near-Term  and  Far-Term 
MPE's  must  take  into  consideration  DMA's  capability  to  absorb 
the  new  technology  without  affecting  the  production 
environment . 

19.1.1  Experimental  Evaluation  Systems 

The  initial  Near-Term  and  Far-Term  HPE  systems  at  each  DMA 
center  will  be  developed  and  introduced  as  an  experimental 
systems.  One  VAX  should  be  acquired  for  delivery  to  the  sys¬ 
tem  developer  and  this  system  in  turn  will  be  used  as  a 
prototype  for  the  experimental  configurations.  These  con¬ 
figurations  will  then  be  evaluated  on  two  VAX's  resident  at 
the  DMA  centers  which  will  serve  as  test-beds  for  the 
developer's  proposed  methods  of  tool  utilization.  DMA  will 
be  required  to  select  a  group  of  personnel  at  each  center  to 
act  as  evaluators  of  the  methodologies  proposed  by  the 
developer  within  the  DMA  environment.  A  separate  task  of  the 
qroup  will  be  to  participate  in  the  development  of  materials 
and  scenarios  for  use  in  training  production  personnel  on  the 
systems. 

Near-Term  and  Far-Term  full-scale  production  MPE  system 
design  will  be  parallel  efforts  to  the  experimental  systems 
implementations  and  evaluations.  To  aid  these  efforts  the 
transition  plan  has  an  underlying  cycle  in  which  as  developer 
generates  a  methodology  for  utilization  of  a  portion  of 
either  the  Near-Term  or  Far-Term  environments;  the  evaluation 
team  analyzes  the  methodology  within  the  DMA  production 
environment;  the  developer  incorporates  changes  as  necessary; 
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and  training  aaterials  and  scenarios  for  the  production  lear- 
Tere  and  Far-Ters  HPE's  are  upgraded.  The  following  nar¬ 
rative  presents  the  eajor  tasks  and  eilestones  of  the 
developer.  A  detailed  schedule  of  the  tasks  and  eilestones 
is  presented  in  Figure  19.1 


In  developing  the  cost  benefit  analysis  for  the  DMA  MPE,  we  identified 
the  requirement  to  mke  tool  sets  available  to  DMA  personnel  during  Phase  I. 
Cost  constraints  for  the  Engineering  Prototype  development  dictated  that 
we  only  cost  out  one  tool  set  at  the  contractor's  facility.  However,  we 
recommend  that  two  tool  sets  (one  at  each  center) ,  plus  maintenance  for 
two  years,  be  included  in  Phase  I  costs  to  improve  tool  access  by  DMA 
personnel.  This  approach  is  necessary  to  avoid  remote  access  problems 
caused  by  ccmnunication  links  to  remote  facilities.  These  problems  were 
very  detrimental  to  tool  evaluations  accomplished  during  the  contract.  This 
approach  while  not  costed  out  in  the  reports,  is  certainly  a  more  feasible 
way  to  accomplish  prototyping  at  the  DMAHTC  and  DMAAC  sites.  It  is  also 
required  that  these  tool  sets  be  hosted  on  VAX's  that  are  identical  to  the 
contractor  site.  If  this  cannot  be  accomplished  under  current  DMA  procure¬ 
ments,  we  reccnmend  that  the  two  VAX's  already  identified  to  support  Ada 
be  added  to  the  Phase  I  funding  profile  to  ensure  successful  prototype 
implementation .  These  tool  bearing  hosts  can  still  be  used  for  Ada  support 
during  Phase  IIA  of  the  program. 
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PHASE  II.  (0  VAX'S) 

o  DESIGN  6  IMPLEMENT  FAR-TERM 
EXPERIMENTAL  SYSTEM 
O  DESIGN  FAR-TERM  FULL-SCALE 
SYSTEM 

o  UPGRADE  METHODOLOGY 
o  IDENTIFY  RED  EFFORTS 
o  SOFTWARE  TOOL  INTEGRATION 

PHASE  IIA.  (2  VAX'S) 


o  IMPLEMENT  FULL-SCALE  SYSTEM 
o  TRAINING 

o  UPGRADE  METHODOLOGIES 
AND  STANDARDS 


Figure  19.1  Transition  Schedule 


19.1.2  Chronological  Tasks 


After  delivery  of  the  initial  VAX  to  the  system  developer, 
the  development  of  the  VAX  based  tool  utilization  methodology 
begins.  As  the  methodology  is  evolved  for  each  tool  a  copy 
of  the  tool  is  distributed  and  hosted  at  each  center  on  their 
experimental  system.  Once  the  methodologies  for  the  tools 
have  been  developed,  evaluated,  and  refined  they  must  be  in¬ 
tegrated  into  a  complete  life  cycle  software  development 
process.  This  process  must  then  be  merged  with  existing  DMA 
standards.  Specifically,  a  4  volume  set  of  standards  is  cur¬ 
rently  under  development  at  DMA.  Indications  are  that  there 
will  be  little  or  no  conflict  between  these  standards  and  the 
recommended  MPE,  however,  requirements  for  enhancements  to 
the  standards  may  be  identified  during  the  work  with  the  ex¬ 
perimental  system.  Near-Term  MPE  development  will  now  be 
complete. 

Implementation  of  the  full-scale  Near-Term  production  MPE 
systems  will  now  proceed.  Six  VAX's  are  to  be  delivered  two 
at  a  time  (one  to  each  center) ,  during  an  estimated  eighteen 
month  period,  to  be  used  as  software  production  computers. 
Additional  implementation  activities  will  include 
methodology/standards  upgrades,  production  training,  network¬ 
ing  of  Near-Term  MPE  VAX's  within  each  center,  and  developing 
communication  links  between  VAX's  and  the  DNIVAC  production 
mainframe. 

Following  the  implementation  of  the  full-scale  Near-Term  MPE 
environment  work  will  begin  on  the  development  and  evaluation 
of  the  Far-Term  MPE.  The  developer's  VAX  will  again  be  used 
as  a  prototype  system  to  be  used  in  the  development  of  ex¬ 
perimental  systems  at  the  DMA  centers.  However,  the 
developer's  original  VAX  computer  will  still  be  used  to  sup¬ 
port  training,  engineering  evaluation,  and  updates  for  the 
Near-Term  production  environment  by  downloading  to  the  Near- 
Term  environment  as  required  from  the  evolving  Far-Term  sys¬ 
tem  prototype.  The  original  experimental  VAX  computers  at 
each  center  will  be  phased  from  the  Near-Term  to  Far-Term 
configurations.  The  development,  implementation,  and  inte¬ 
gration  of  the  Far-Term  MPE  with  the  DMA  standards  will  be 
similar  to  the  Near-Term  development.  However,  Far-Term 
development  and  implementation  will  benefit  from  the  Near- 
Term  transition  experience.  The  resident  Near-Term  produc¬ 
tion  systems  can  then  be  converted  to  the  Far-Term  MPE  and 
the  transition  will  be  complete. 

19.2  TRANSITION  SUPPORT  RECOMMENDATIONS 
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Two  recommendations  which  evolved  from  the  Tool  Survey  task 
are  to  provide  long-term  formal  training  to  personnel  prior 
to  injecting  new  technology;  and  to  train  less  experienced 
personnel  within  the  production  environment  first. 
Additionally,  MPE  administrator  and  toolsmith  functions  would 
be  beneficial  in  supporting  the  transition. 

The  MPE  administrator  and  toolsmith  functions  would  be  sup¬ 
port  positions  which  would  primarily  serve  as  the  focal  point 
for  management  to  observe  the  system  activities  and  as  an  in¬ 
formation  source  for  MPE  training.  Specifically,  the  MPE  ad¬ 
ministrator  would  be  responsible  for  an  overall  understanding 
of  the  MPE  and  its  i'se.  The  toolsmiths  would  aid  the  MPE  ad¬ 
ministrator  and  would  each  be  responsible  for  a  thorough 
knowledge  of  a  particular  component  of  the  MPE  system. 
Personnel  involved  with  the  functions  would  be  knowledgeable 
in  the  current  tools  and  methodologies  contained  in  the  MPE 
as  well  as  the  VAX  environment  on  which  it  would  run.  Tasks 
would  include  performing  error  rate  studies,  helping  users 
with  software  development  problems  and  the  identification  of 
needs  not  satisfied  within  the  user/management  communities. 
The  personnel  staffing  this  function  should  be  located  close 
to  the  MPE  terminal  areas  in  order  to  encourage  programmers 
to  bring  their  problems  immediately  instead  of  rerunning 
several  times  before  giving  up.  The  MPE  administrator  and 
toolsmith  personnel  would  not  be  expected  to  debug  user’s 
programs  but  would  be  expected  to  help  all  users  who  had  sof¬ 
tware  development  run  problems  using  the  MPE. 

The  transition  plan  provides  for  four  months  of  training  on 
each  subset  of  the  MPE.  The  recommendations  concerning 
training,  that  it  be  long-term  and  with  less  experienced  per¬ 
sonnel  first,  are  not  the  only  recommendations  which  can  be 
generated  from  the  tool  survey  results.  The  major  problem 
encountered  during  the  evaluation  task  of  the  tool  survey  was 
schedule  impacts  due  to  access/hardware  problems  using  remote 
equipment.  This  data  strongly  supports  training  DMA  person¬ 
nel  on-site  rather  than  using  communication  links  to  remote 
computers.  Another  problem  was  the  training 
scenario/materials  were  not  adequate  due  to  problems  of  in¬ 
terpretation  and  development  time.  The  transition  plan 
provides  for  six  months  to  develop  a  scenario  and  materials 
with  DMA  cooperation  prior  to  training  being  accomplished  on 
each  software  subset  or  system  of  the  MPE.  A  final  recommen¬ 
dation  from  the  tool  survey  evaluation  task  is  that  the 
groups  being  trained  be  no  larger  than  seven  people  to  allow 
for  proper  interaction  between  instructor  and  group  and  amon¬ 
gst  the  group. 
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20.0 


COST_BENEFITS_ANALYSIS 


An  important  aspect  of  the  Defense  Mapping  Agency  Modern 
Programming  Environment  specification  is  its  cost.  This  sec¬ 
tion  contains  the  DMA  MPE  cost  benefits  analysis.  The  objec¬ 
tives  of  this  cost  benefits  analysis  are: 

Objective  1:  To  estimate  the  total  cost  for  the  develop¬ 
ment  and  implementation  of  the  DMA  MPE  as  specified  in 
this  document. 

Objective  2:  To  estimate  the  savings  that  can  be 

realized  by  using  the  DMA  MPE  as  compared  to  the  con¬ 
tinued  use  of  existing  DMA  methods  of  software  develop¬ 
ment  and  maintenance. 

Objective  3:  To  predict  the  return  on  investment  over  a 
ten  year  time  span  starting  from  the  beginning  of  MPE 
development. 

To  accomplish  these  objectives  the  following  assumptions  were 
made: 

1)  The  MPE  development  and  implementation  task  begins  in 
July  1983  and  ends  in  December  1987. 

2)  Hardware  and  software  tool  systems  will  be  maintained 

by  the  manufacturers  or  vendors  via  maintenance 

agreements. 

3)  Government  furnished  software  used  in  the  MPE  is 

available  to  DMA  at  essentially  no  cost. 

In  addition,  the  Defense  Mapping  Agency  provided  certain  data 
particular  to  their  organization.  This  data  was: 

1)  Annual  inflation  rate  of  5*, 

2)  Overtime  burden  rate  of  1.9*,  and 

3)  Typical  DMA  workyear  cost  of  $25,000. 

Figure  20.1  shows  the  major  components  in  the  cost  analysis 
of  the  DMA  MPE. 
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Block  Diagram  of  DMA  Modern  Programming 
Environment  Cost  Analysis 


Por  the  purposes  of  cost  estimating.  Figure  20.2  shows  the 
top-level  configuration  of  the  DMA  MPB  VAX  computers.  The 
solid  lines  indicate  equipments  that  will  be  purchased  or 
developed  and  hence  their  cost  must  be  estimated.  Those 
equipments  indicated  by  dashed  lines  are  already  existing  at 
the  centers.  The  detailed  cost  of  each  particular  hardware 
and  software  equipment  item  is  shown  in  Figure  20.3,  the  unit 
cost  table. 


MODERN  rROCRAMHNC  ENVIRONMENT  CMPE) 


AT  DMAAC: 


AT  DMAHTC : 


AT  CONTRACTOR 
SITE: 


Solid  lines  represent  hardware  purchased  In  this 
DMA  MPE  development  effort  (9  VAX’s) 


Dashed  lines  represent  existing  hardware  at  DMA 


Figure  20.2  TOP  LEVEL  CONFIGURATION  FOR  DMA  MPE  VAX  COMPUTERS 


161 


.■If-..  . 


TOOL 

HARDWARE/ 

SOURCE 

OBJECT 

MAINTENANCE 

ANNUAL 

LEASE/RENT 

USE. IT 

_ 

(11) 

$142,000 

(2) 

$ 

10,650 

(1) 

$  2,500 

(5) 

IS/1  PWB 

$ 

43,000 

$ 

31 ,500 

( 2  )  ( 4 ) 

$ 

7,500 

(1  ) 

- 

(7) 

IS/1  INed 

$ 

30,000 

$ 

6,500 

(2) 

$ 

1,250 

(1) 

- 

(7) 

IS/1  INword 

$ 

20,000 

$ 

8,000 

(2) 

$ 

1,500 

(1  ) 

- 

(7) 

S.A.I.  SDDL 

$ 

20,000 

$ 

5,000 

$ 

25 

(12) 

HYPERGRAPHICS 

N/A 

$ 

500 

(2 )  (4 ) 

N/A 

N/A 

UNIX  LICENSE 

N/A 

_ 

(4) 

N/A 

N/A 

VAX  11/780(6) 

$274 , 900 

N/A 

$ 

1,349 

(8) 

N/A 

TERMINALS) 

$ 

2,700 

N/A 

$ 

243 

(1) 

N/A 

TERMINAL ( 1 0 ) 

$ 

2,400 

N/A 

$ 

22 

(8) 

N/A 

MULTITERMINAL 

$ 

8,100 

(4) 

N/A 

INCLUDED 

N/A 

EMULATOR 

COMMUNICATIONS 

$ 

1,575 

N/A 

$ 

12 

(8) 

N/A 

DEVICE 

ASYNCHRONOUS 

$ 

6,500 

N/A 

$ 

84 

(8) 

N/A 

MULTIPLEXERS 

NETWORK 

- 

(3) 

N/A 

$ 

69,500 

( 1 )  ( 3 ) 

$375,000 

(1) 

SYSTEM 

FAVS 

- 

(3) 

N/A 

INCLUDED 

N/A 

NETWORK  LINK 

$ 

4,400 

N/A 

$ 

39 

(8) 

N/A 

PROTOCOL 

- 

(13) 

- 

(13) 

(13) 

(13 

DISK  PACK 

$ 

1,500 

N/A 

N/A 

N/A 

TAPE(2400) 

$ 

30 

N/A 

N/A 

N/A 

VUE 

$ 

13,500 

N/A 

INCLUDED 

(1) 

N/A 

,1)  DATA  FOR  FIRST  YEAR  ONLY 

(2)  SINGLE  COMPUTER  COST  -  DISCOUNTS  AVAILABLE  FOR  ADDITIONAL  LICENSES 

(3)  ALREADY  RESIDENT  AT  DMA 

(4)  PERPETUAL  LICENSE 

(5)  SHORT  TERM 

(6)  INCLUDES  O/S,  CONSOLE,  FLOPPY  DISK,  2MB  MEMORY,  TAPE  i  DISK  DRIVES 

(7)  MUST  INCLUDE  HARDWARE 

(8)  MONTHLY 

(9)  INtext  II 

(10)  VT102 

(11)  SOURCE  MAY  BE  PUT  IN  ESCROW 

(12)  PER  UPDATE 

(13)  CAPABILITY  EXISTS  BUT  MUST  USE  CUSTOMIZED  SOFTWARE 


Figure  20.3 


Unit  Cost  Table 


The  data  in  this  unit  cost  table  represents  the  most  current 
costs  at  the  time  of  the  preparation  of  this  report.  This 
data  was  aggregated  into  five  cost  estimating  factors  that 
were  used  to  complete  the  cost  estimate  for  the  DMA  MPB. 


These  factors  are: 

1)  Hardware  purchases  for  one  system  -$393K 

2)  Software  purchases  for  one  tool  set  -$315K 

3)  Hardware  maintenance  costs  for  one  system 

for  one  year  -$  26K 

4)  Software  maintenance  costs  for  one  tool 

setfor  one  year  -$  27K 

5)  Average  cost  of  one  workyear  of  contractor 

labor  -$  60K 


To  estimate  the  total  cost  for  the  DMA  MPE,  the  first  objec¬ 
tive  of  the  cost  analysis,  recurring  and  non-recurring  costs 
were  identified.  Recurring  costs  included  hardware 
maintenance,  software  maintenance,  and  personnel  needed  to 
directly  support  the  MPE  operation.  Non-recurring  costs  in¬ 
cluded  hardware  purchases,  software  purchases,  and  contractor 
development  labor.  The  non-recurring  costs  were  confined  to 
the  the  DMA  MPE  development  time  span  (July  1983  to  December 
1987)  ;  however,  the  recurring  costs  with  inflation  included 
were  distributed  from  July  1983  to  December  1993  to  be  con¬ 
sistent  with  the  return  on  investment  estimate  (objective  3)  . 
A  tabulated  cost  for  the  DMA  MPE  was  completed  using  this 
schedule  data  and  the  cost  estimating  factors.  Figure  20.4 
shows  the  total  cost  estimate  for  the  DMA  MPE  by  phase,  fund¬ 
ing  source,  and  fiscal  year.  The  total  cost  of  the  DMA  MPE 
is  approximately  $11  million.  Figure  20.5  shows  how  each 
fiscal  year  entry  in  Figure  20.4  is  further  partitioned  into 
hardware,  software,  and  labor.  Hardware  in  this  context 
means  both  hardware  purchases  and  hardware  maintenance,  and 
similarly  for  software.  Hardware  costs  approximately  $4.2 
million,  software  costs  approximately  $2.5  million,  and  con¬ 
tractor  labor  costs  approximately  $4.3  million. 
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TOTALS 


COSTS  (In  Thousands 


20.5  Cost  Estimates  for  DMA  MPE  by  Phase,  Resource 


Tha  second  cost  analysis  objective,  the  savings  realized  by 
using  the  DMA  MPE,  was  completed  by  modeling  the  Defense 
Mapping  Agency's  usage  of  the  Modern  Programming  Environment 
capabilities-  First,  data  obtained  from  the  General  Dynamics 
questionnaire  conducted  in  Stage  1  of  this  study  was  analyzed 
to  estimate  the  percentage  of  total  DMA  software  efforts  that 
is  spent  in  each  software  life  cycle  phase.  Next,  an  es¬ 
timate  of  the  productivity  improvement  for  each  DMA  MPE  sof¬ 
tware  tool  was  made.  These  two  data  were  combined,  and  an 
estimated  productivity  improvement  due  to  the  DMA  MPE 
capabilities  of  approximately  40*  was  calculated.  Finally, 
the  estimated  size  of  the  DMA  programming  population,  an  es¬ 
timate  of  the  rate  of  productivity  improvement  realization, 
and  the  DMA  workyear  costs  with  inflation  were  used  to  cal¬ 
culate  an  expected  yearly  dollar  savings  caused  by  the  DMA 
MPE.  The  results  of  this  calculation  are  shown  in  Figure 
20.6. 
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To  calculate  the  return  on  investment  (objective  3)  the 
tabulated  yearly  cost  of  the  DMA  MPE  (the  results  from  objec¬ 
tive  1)  vas  subtracted  from  the  tabulated  yearly  savings  of 
the  DMA  MPE  (the  results  from  objective  2) .  This  net  savings 
tras  accumulated  and  plotted  in  Figure  20.7.  The  cumulative 
net  savings  is  the  sum  of  the  yearly  savings  minus  yearly 
costs.  Note  that  the  total  cost  of  the  DMA  MPE  will  be 
recovered  after  five  years  (in  1988) ,  and  after  ten  years  (in 
1993)  an  estimated  cumulative  net  savings  of  $25  million  will 
be  realized.  This  represents  an  excellent  return  on  the 
initial  investment  for  the  Defense  Mapping  Agency  Modern 
Programming  Environment. 
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CUMULATIVE  NET  SAVINGS  (IN  MILLIONS  OF  DOLLARS) 


Figure  20.7 


Graph  of  Cumulative  Net  Savings  for  DMA 
MPE  Development 


- 
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22.0  LIST  OF  ABBREVIATIONS 


ADP  AUTOMATED  DATA  PROCESSING 

ADS  AUTOMATED  DATA  SYSTEM 

AIAA  AMERICAN  INSTITUTE  OF  AERONAUTICS  AND 

ASTRONAUTICS 

APSE  ADA  PROGRAMMING  SUPPORT  ENVIRONMENT 

ASCII  AMERICAN  STANDARD  CODE  FOR  INFORMATION  INTERCHANGE 

CDC  CONTROL  DATA  CORPORATION 

CDRL  CONTRACT  DATA  REQUIREMENTS  LIST 

CIE  CONCEPT  IMPLEMENTATION  EVALUATION 

CPU  CENTRAL  PROCESSING  UNIT 

CRT  CATHODE  RAY  TUBE 

DEC  DIGITAL  EQUIPMENT  CORPORATION 

DLMS  DIGITAL  LAND  MASS  SYSTEM 

DMA  DEFENSE  MAPPING  AGENCY 

DMA AC  DEFENSE  MAPPING  AGENCY  AEROSPACE  CENTER 

DMAHQ  DEFENSE  MAPPING  AGENCY  HEADQUARTERS 

DMAHTC  DEFENSE  MAPPING  AGENCY  HYDROGRAPHIC/ 

TOPOGRAPHIC  CENTER 

EDSC  EASTERN  DATA  SYSTEMS  CENTER 

FAME  FRONT  END  ANALYSIS  AND  MODELING 

ENVIRONMENT 

FAVS  FORTRAN  AUTOMATED  VERIFICATION  SYSTEM 

FEDSIM  FEDERAL  COMPUTER  PERFORMANCE  AND  EVALUATION 

AND  SIMULATION  CENTER 

GD/DSD  GENERAL  DYNAMICS/DATA  SYSTEMS  DIVISION 

HOL  HIGH  ORDER  LANGUAGE 

HOS  HIGHER  ORDER  SOFTWARE 

ICPDSS  INTERACTIVE  COMPUTER  PROGRAM  DEVELOPMENT  SYSTEM  STUDY 
IEEE  INSTITUTE  OF  ELECTRICAL  AND  ELECTRONIC 

ENGINEERS 

IPF  INTERACTIVE  PROCESSING  FACILITY 

IPR  IN-PROCESS-REVIEW 

KIPS  THOUSANDS  OF  INSTRUCTIONS  PER  SECOND 

KOPS  THOUSANDS  OF  OPERATIONS  PER  SECOND 

LAN  LOCAL  AREA  NETWORK 

LARE  LOGICON'S  AUTOMATED  REQUIREMENTS  ENGINEERING 

MAPSE  MINIMAL  ADA  PROGRAMMING  SUPPORT  ENVIRONMENT 

HEDS  MULTI-LEVEL  EXPRESSION  DESIGN  SYSTEM 

MPE  MODERN  PROGRAMMING  ENVIRONMENT 

NBS  NATIONAL  BUREAU  OF  STANDARDS 

PRICE-S  PROGRAMMED  REVIEW  OF  INFORMATION  FOR  COSTING  AND 
EVALUATION  -  SOFTWARE 
PSL  PROGRAM  SUPPORT  LIBRARY 

PSL/PSA  PROBLEM  STATEMENT  LANGUAGE/PROBLEM  STATEMENT  ANALYZER 
PWB  PROGRAMMER ' S  WORK  BENCH 

RADC  ROME  AIR  DEVELOPMENT  CENTER 

RAT  RESOURCE  ALLOCATION  TOOL 

RSL/REVS  REQUIREMENTS  STATEMENT  LANGUAGE/REQUIREMENTS  ENGINEERING 
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VALIDATION  SYSTEM 
R JE  REMOTE  JOB  ENTRY 

SCMS  SOFTWARE  COMPLEXITY  MEASUREMENT  SYSTEM 

SDDL  SOFTWARE  DESIGN  AND  DOCUMENTATION  LANGUAGE 

SES  SOFTWARE  ENGINEERING  SYSTEM 

SIP  SOFTWARE  IMPROVEMENT  PROGRAM 

SON/SOC  STATEMENT  OF  OPERATION  NEED  &  SYSTEM 
OPERATIONAL  CONCEPT 

SRIMP  SOFTWARE  REQUIREMENTS  INTEGRATED  MODELING  PROGRAM 

TBD  TO  BE  DETERMINED 

TBH  TOOL  BEARING  HOST 

TEP  TOOL  EVALUATION  PLAN 

TTY  TELETYPE 

TX  TEXT  EDITOR 

UIFOLA  USER  INTERFACE  FOR  ON  LINE  ASSISTANCE 

UNCL  UNCLASSIFIED 
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APPENDICIBS 


This  section  contains  the  following  10  appendicies: 


Appendix  A 
Appendix  B 
Appendix  C 
Appendix  D 
Appendix  E 
Appendix  F 
Appendix  G 
Appendix  H 


Appendix  I 
Appendix  J 


DMA  SURVEY  QUESTIONNAIRE 
DEMONSTRATION  RESPONSE  FORM 
EVALUATION  TOOL  SET 
LIFE  CYCLE  QUESTIONNAIRES 
EVALUATION  SURVEY  RESPONSES  SUMMARIZED 
EVALUATION  ACTIVITY  STATISTIS 
CONCEPT  IMPLEMENTATION  EVALUATION  MATRIX 
SUMMARY  STATISTICS  FROM  EVALUATION 
CIE  BY  NEED 
CIE  BY  TOOL 
CIE  BY  SCORE 
CIE  BY  CONCEPT 
BEST  CASE  BY  NEED 
BEST  CASE  BY  CONCEPT 
BEST  CASE  BY  MODERN  PROGRAMMING 
ENVIRONMENT  FOR  DMA 
LORAN  NAVIGATIONAL  LATTICE  PROBLEM 
CONCEPT  IMPLEMENTATION  EVALUATION  SHEETS 
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APPENDIX  A 

DMA  SURVEY  QUESTIONNAIRE 
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GENERAL  DYNAMICS  DMA  SURVEY 


Overview : 

The  Defense  Mapping  Agency  (DMA)  is  involved  in  a  study  to 
design  a  modern  programming  environment  computer  system.  To 
effectively  produce  this  system,  DMA  must  develop  or  obtain 
support  programs/tools  and  accompanying  procedures  for 
product  software  development.  Current  projects  frequently 
employ  specialized  support  software  packages  unique  to  each 
development.  It  is  evident  that  significant  benefits  can  be 
realized  if  a  common  "core"  of  tools  and  procedures  can  be 
developed  for  all  DMA  centers. 

The  first  phase  of  the  investigation  is  a  study  of  the  needs 
of  DMA  and  the  tools  which  are  available,  or  could  be 
developed,  to  satisfy  those  needs.  While  the  study  is 
directed  at  production  software,  many  of  the  tools  being  in¬ 
vestigated  have  a  wider  potential  and  other  areas  such  as 
technical  support  software  will  also  be  considered. 

A  questionnaire  has  been  developed  to  assist  in  the  study 
phase.  It  will  help  to  determine  needs  at  DMA  and  will  help 
identify  currently  used  tools  appropriate  for  common  use. 
This  questionnaire  will  also  function  as  a  tool  in  validating 
the  findings  of  the  Boeing  report,  RADC-TR-79-343,  as  well  as 
attempt  to  gather  information  about  the  future  plans  of  DMA 
in  the  areas  of  operations  and  policies. 

Your  aid  in  identifying  needs  is  appreciated  and  should  lead 
to  a  system  capable  of  supporting  these  needs  in  a  cost  ef¬ 
fective  manner.  The  questionnaire  is  only  a  beginning  point 
for  the  study.  Personal  contacts  will  follow  to  clarify 
findings  and  to  allow  for  additional  inputs. 

There  are  five  parts  to  the  questionnaire.  Each  person  will 
be  asked  to  answer  three  sections.  The  respondent  section 
will  be  used  to  correlate  answers  with  respect  to  a  person's 
background.  A  tools  section  is  included  to  gather  general 
knowledge  about  what  software  tools  are  and  their  usefulness. 
One  of  three  other  sections  will  also  be  answered  :  1)  a 

technical  section  to  gather  data  on  operations  2)  a 

management  section  to  determine  methods  of  operation  3)  a 

policies  section  on  DMA  planning,  control,  organization  and 
direction. 

If  you  need  additional  space  to  respond  to  a  question  please 
attach  extra  sheets  and  indicate  question  section,  number  and 
letter  as  applicable. 
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DATE. 


ORGANIZATION 
PHONE  NO. _ 


Answer  each  question  as  it  pertains  to  software/hardware  in 
your  organization.  Hhen  you  don't  feel  qualified  to  respond 
to  a  question,  please  indicate  this  in  the  space  left  for 
coanents.  The  coaaent  space  should  also  be  used  for  any  ad¬ 
ditional  inforaation  that  you  feel  is  pertinent  to  the 
question.  Please  leave  response  areas  blank  if  inforaation 
is  not  known  except  when  'unknown'  is  an  answer. 

Organization  DHAHQ _ DHAHTC _ D8AAC _ 

1.  Respondent  characteristics: 


A.  Position  description: _ 

B.  Current  project  assignaent : _ _ _ _ _ _ _ _ 

C.  Total  years  experience  in  each  category : (check  correct  range) 

Technical  Hanagerial  DBA 

0  -  1/2  :  _  _ 

1/2  -  1  1/2: 

1  1/2  -  3  :  _  _  _ 

3-5  :  _____  _  _ _ 

Over  5  :  _ 


D.  Acadeaic  background: 

Field  Add.  hours 

_ Associate  _  _ _ 

_ Bachelors  _ _  _ 

_ Rasters  _  _ _ _ 

_  Doctoral  _  _ 

_ _ Post  Graduate  _  _ 


E 


List  any  OJT  schools/seainars/classes  attended: 


List  languages  you  are  currently  using: 


List  other  languages  with  which  you  have  had 
experience : _ 


Computer (s)  you  are  currently  using: 
Using  Years  exp. 


Computer  (s)  you  have  used  in  the  past: 
Used  Years  exp. 


Comments: 
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PAST  I:  Technical 


1.  Project (s)  Description.  (This  information  should  be  general  with 
respect  to  single/aultiple  projects  being  currently  perforaed.) 

A.  Indicate  percentage  of  project  (s)  perforaed  in  following  nodes: 

_ _  Interactive 

_  Batch 

B.  Estimate  the  percentage  of  work  being  perforaed  and  the 
responsible  agency  in  the  following  life  cycle  (s) .  Percentages 
should  only  apply  to  DMA  tasks. 

Percentage  DBA  Subcontracted 

Bequireaents  _  _  _ 

Design  _  _  _ 

Coding  _  _  _ 

Testing  _  _  _ 

Maintenance  _  _  _ 

C.  Computer  (s)  in  use  is  _ 

Is  reentrant  code  used  for  system  processing (common  banks)? 

Yes _  No _  Don •  t  know _ 

Scheduling  is  accoaplished  _ automatically  or  _ manually. 

Indicate  any  of  the  following  denand  systems  being  used: 

_  Conversational  Tiae  Sharing (CTS) 

Editor  (ED) 

Symbolic  Streaa  Generator (JCL) 

Pull  Screen  Editor (PSE) 

Other  _ _ 


what  configuration  aanageaent  systea  for  change  control  of 
production  programs  is  in  use? _ . 

D.  Language  (s)  being  used  is  _ 

E.  On  which  aediua  does  the  application  source  prograa  code  (s) 
reside? 

Cassette 

Hardcopy  _ 

Unknown  _ ~ 

Cards 

Tape  _ 
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F.  Hon  is  computer  accessed? 

Over-the-counter  _ 

Beeote  reader  _ 

Reaote  terainal  _ 


G.  How  is  output  received? 

on-line  application  systea  _ 

Tape 

Reaote  printer  _ ~~ 

nicrofila  —  - 

Over-the-counter 

Renote  terainal  _ ~~ 

H.  How  aany  technical  personnel  are  involved  with 

the  task  (s) ?  k  range  aay  be  given  for  multiple  projects. 


I. 


J. 


what  is  expected  turnaround  tiae  in  hours? 

what  is  actual  turnaround  tiae  in  hours?  _ 

Pill  in  the  blanks  with  characteristics  which  apply. 

1.  Require  about _ R  words  of  central  aeaory. 

Execution  in  approxiaately _ CPO  seconds. 


2- 

3. 

4. 

5. 


Requires _ (nuaber  of)  secondary  storage  devices. 

tape _ disk _ drua _ 

Contains _ executable  lines  of  code. 

Docuaentation  produced: 


Functional  rqats.  desc. 
Data  rqats.  document 
Systen/subsystea  spec. 
Prograa  specification 
Data  base  specification 
User’s  aanual 
Test  plan 


Always  Often 


seldom 


Never 


Consents 
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K .  Software  is  generally  developed 

_ for  a  one-tiee  application  by  a  single  user. 

_ for  application  by  a  few  users. 

_ for  application  by  eany  users  (production  software). 

_ ~for  application  by  few  users,  yet  in  reality  is  used  by  eany. 

Coaaents 


L.  Development  aids 

Osed  Paailiar  Naae/coaaent 

Tape  Management  System 
MAP  processors 
Compilers 
Assemblers 
Linkage  editors 
Text  Editors 

Configuration  Control  Aids 
Other  _ 

H.  Environment? 


Do  all  qualified  individuals  have  access  to  the  computer (s) ? 
Is  there  a  specialist  staff  assigned  for  computer  access? _ 

O.  Security? 

Source  Data 

Unclassified  _  _ 

Confidential  _  _ 

Secret  _  _ 

Top  Secret  _  _ 

Is  access  to  your  physical  area  limited  (e.g.  SCI)? _ 

P.  Define  the  crew  that  will  be  working  on  your  project  (s)  : 

Extensive  experience  -  some  top  talent  _ 

Normal  crew  -  experienced  _ 

Mixed  experience  -  some  new  hire  _ 

Relatively  inexperienced  -  many  new  hire  _ 

Q.  How  familiar  is  the  project  (s)  to  your  organization? 

Rework  of  previous  project  _ 

Paailiar  type  of  project  _ 

Normal  new  project  _ 

No  previous  experience  _ 
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Are  there  any  factors  that  coaid  coeplicate  this  project? 

Mew  hardware  _ 

Hew  language  _ _ 

Single  source  developaent  _ 

Parallel  to  hardware  developaent  _ _ 

Changing  requirements  _ 

State-of-the-art  advanceaent  _ 

Other _  _  _ 


Is  a  project  generally  integrated  into  a  larger  system? 
If  yes,  is  this  a  typical  integration  task  for  your 
organization?  _______ 


Check  area  (s)  which  are  typical  of  your  project  (s) : 

_  Editing 

Word  processing 

_  Electronic  aail/conferencing 

Hatheaatical 

_  String  manipulation 

_  Data  storage  and  retrieval 

Real-time  command  and  control 

_  Interactive  operations 

_  Operating  system 

_  Other  _ 

Comments? 


y'fe-Si  -if 


2 


Work  Environment 


A.  Is  your  work  dependent  upon  previous  work 

accomplished? _ 

Is  your  work  used  by  someone  else  when 

complete? _ 

Comments _ _  _ 


B. 


c. 


Check  if  any  item  is  used  on  your  projects. 

Notebook _ 

Formal  walkthroughs _ 

Informal  team  sessions _ 

Formal  reviews _ 

Training  new  personnel _ 

Standards/guidelines _ _ 

Backup  person  assigned  for  tasks _ 

Trouble/problem  reports _ 

Configuration  management _ 

Coding  style  guide _ 

Testing  worksheets _ 


Check  the  statements  that  apply  to  the  way  manhours,  computer 
usage  and  schedule  estimates  are  derived: 


1 .  formula  (s) 

2.  project  comparison 

3.  individual  expertise 
u.  dictated  by  user 

5.  simulation 

6.  informally 

7.  Other 


Always  Often  Seldom  Never 


D.  Check  the  statements  that  apply  to  resource  planning: 

Always  Often  Seldom  Never 

1.  Guidelines  used  in  plan 

preparation  _  _  _ 

2.  Customer/user  partici¬ 
pation  _  _  _ 

3.  Resources  allocated  by  ~~~ 

project  _  _  _ 

4 •  No  formal  planning 

(guesstimates)  _  _  _ 
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Is  the  need  for  special  skills/expertise  recognized  and  addressed 
in  the  project  plan? 


Always  Often  _ Seldom  never 


Consents 


P.  Rank  those  iteas  which  are  likely  reasons  for  rewrite  or  change 
of  the  software  reguirenents  daring  the  development  effort. 

(1  -  nost  likely,  4  =  least  likely). 

_  specification  errors 

_  prograaaer  errors 

_  anbiguous,  incoaplete  or  inconsistent  specifications 

_  change  in  project  reguirenents 

_  user  and/or  developer  becoae  better  informed 

_ _  other: 


3.  During  the  software  planning  process,  are  interaediate  goals 
(project  milestones)  defined?  _ YES _ MO 

a.  If  YES,  are  these  milestones  used  to  monitor  project 
progress? 

_ Always  _ _ Often _ _ Seldom  Never 


Comments 


I. 


Formal  (documented)  procedures  to  be  followed  during  reguirenents 
specification 

_  do  not  exist 

_  exist  and  are  followed 

_  _  exist  but  ate  not  followed 


The  outcome  of  the  reguirenents  definition  effort  is: 

Always  Often  Seldom  Never 

1.  a  formal,  documented 
and  approved  reguire- 

ments  specification  _  _  _ _ 

2.  an  informal  agreement 

with  user/customer  _  _  _____ 

3.  a  loosely  defined  set  of 
reguirenents  which  is 
subject  to  change  during 

project  development  _  _  _  _____ 

4.  Other 
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J.  The  design  effort  is  complete  when: 

Always  Often  Seldos  Revet 

1.  assurance  is  given 
that  all  reguiresents 

have  ';een  addressed  _ _  _  _ _  _ 

2.  the  scheduled  due  date 
for  design  completion 

is  reached  _ _ _  _ _ _ 

3.  the  next-lower  level  of 
design  would  result  in 
iapleaentation  decisions 

4.  The  user  has  reviewed 

and  approved  the  design  _  _  _  _ 

K.  Check  any  design  techniques  used. 

_  Structured  coding 

_  Walkthroughs 

_  Peer  reviews 

_  Top-down  design 

Naming  conventions 

_  nodular  coding 

_  Data/Pile  foraatting 

_  commenting  conventions 

_  Design  language 

Pseudo-code 
Flowcharts 
H IPO  charts 

_  Other: _ 

L.  Proa  the  list  below,  check  the  tools  which  are  used  during 
software  development  and  indicate  your  opinion  of  their  useful¬ 
ness  and  availability  (Y  =  yes,  N  =  no). 

USED  SUPPORTED  AVAILABLE  DOCUHENTED 


SNOOPY 

PHD 

Dynaaic  Dump  Routines 

FLAP 

INDEX 

Univ  of  HD  Text  Editor 

PILESCAN 

FOBFLO 

TIDY 

SIP/PAR 

TIP/DHS 

REPORRATTER 
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STRUCT8AW-1 

PA»S 

PSL 


List  ths  operating  systea  utilities  (i.e.,  DOWNDATEB ,  PPCK, 

UPDO,  TDOhp,  SORTSDP,  PLIST,  LABEL,  VTBAI,  etc.)  eost  frequently 
used  in  your  organization. 


Coaeents 


H.  The  reporting  aechanisas  used  within  projects  are: (indicate 
with  a  T  for  verbal  or  M  for  sechanical) 

_  Weekly  status  fora 

Milestone  charts 
Technical  aeaos 

_  Inforaal  aeetings 

_  Foraal  reviews 

_  Notebook 

Interactive  nail 

_  Other  _ 

Consents  _  _ _ _  _ 


N.  when  is  software  docuaentation  generally  produced: 

_  As  the  software  is  being  developed 

_  After  the  software  has  been  developed 

_  Only  when  required  by  the  project  plan 


Consents 


0.  At  which  levels  and  by  whoa  do  software  testing  and  evaluation 
occur? 


DESIGNER 

Module  level  _ 

A t  aodule  integration  _ ~ 

At  systea  testing  _  .II 

Other: _  _ 


QA  DSEB 
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p.  Does  completed  software  undergo  quality  assurance  testing  by 
a  group  which  is  independent  of  the  developeent  teaa? 

_ tlways  _ Most  of  the  tiae  _ Soaetiaes  _ Newer 

Coa  aents _ _ _ _  _ _ 


Q.  ire  foraal  test  plans/test  strategies  developed  and 
docuaented? 

_ ilways  _ Host  of  the  tiae  _ Soaetiaes  _ Never 

Do  these  include  testing  to  insure  that  all  requireaents 
have  been  aet? 


_ Always  _ Host  of  the  tiae  _ Soaetiaes  _ Never 

R.  Do  quality  assurance  procedures  or  guidelines  exist? 
_YES  _  NO 


Check  the  activities  to  which  they  apply  and  the  degree  to 
which  they  are  followed: 


Requireaents  specification 
Design  specification 
Coding 

Docuaentation 

Testing 

Haintenance 

Redesign,  code,  retest,  etc. 


Rigidly  Noainally  Not  used 


Coaaents 
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S.  During  the  program  aaintenance,  estiaate  the  percent  of 
effort  spent  in  each  of  the  following: 

Analysis  and  respecification  of  requirements 
Redesign 
Coding 

Retesting  (by  developers) 


Coaaents 


PART  II:  MANAGEMENT 
1.  Software  Development 


Do  you  develop  software  for 
varied  hardware  config¬ 
urations? 

Do  you  encounter  software 
transfer  probleas? 

Do  you  design  software 
to  be  portable? 


Always 


Often 


Seldoa  Never 


Consents 


Describe  the  freguency  of  errors  discovered  in  operational 
software : 

_ Very  low  _ Low  _ Moderately  low 

_  Very  high  _  High  _  Moderately  high 

Comments _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 


Estiaate  the  percent  of  total  effort  devoted  to 

Adding  new  capabilities  to  existing  programs 
Starting  from  scratch  to  produce  a  new  program 
Detection/correction  of  errors  in  existing  production 
pcograas 

Coaaents  _  _ _ _ _  _ _ _ _  _ 


Hank  the  following  as  used  in  project  planning: 

(1  =  not  used;  5  =  often  used) 

Milestone  identification 
Resource  availability 

_  Hanloading 

_ 3  Cost  analysis 

Schedule  criticality 

In  which  area  could  the  aost  iaprovenent  be  achieved? 
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E.  To  what  extent  is  the  user  involved  in  the  developaent 
effort? 

_ _ none  soee  _ adequate  too  such 

P.  What  types  of  reviews  are  used  and  with  whoa  to  track 
developaent? 

Banageaent  Technicians 

_  Walkthroughs  _ 

Design  reviews  ~ _ 

_  Inforaal  aeetings  _  _ ~ 

_ Poraal  reviews  _  ~~ _ 

_  Status  reports  _  _ 

_  Trouble  reports  _  _ _ 

_  Technical  aeaos  _  _ 

Other: _  ”  ~ 


3.  is  developaent  hindered  by  paperwork? _  List  reports 

coaaonly  generated. _ IIIII _ _ 


H.  what  aethods  currently  exist  for  docuaenting  developaent? 

_  Prograa  design  language 

_  Progressing  style  guides 

_  Prograaaing  standards 

Design  specifications 
User's  aanuals 
Configuration  controls 

Other: _ 

which  aethods  are  used?  (Bake  second  check  aark) 

2.  Project  Support 

A.  Bate  the  following  support  activities/roles  in  their 
effect,  upon  project  coapletion/failure 
(H  =  nigh  influence;  L  =  low  influence) : 

Secretary 

_  Keypunch 

Technical  consulting 
Turnaround  tiae 
Bachine  access 
Cosputer  scheduling 
Coaputer  resource  allocation 
Paperwork 
Bevievs 
III  Planning 
_  Training  support 
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B.  Which  are  qualities  of  DBA  software? 

Portable 

_  Bodifiable 

_  Understandable 

C.  Do  guidelines  exist  to  insure  quality  software  develop 

■ent? _ 

Coaaents _  _  _ _ 


D. 


Rate  the  following  as  to  their  importance  in  software 
development.  (1  =  not  important;  5  -  very  important) 


Budget 

_  Schedule 

User  involvement 
Baintainability 

_  Portability 

_  Performance 

Personnel  productivity 
Documentation 

Other;  _ _ _ _ _ _ _ 


E.  What  reports  are  received  on  pro ject  (s) ,  how  often  are  they 
received  and  are  they  verbal  or  mechanical? 


Report 


Frequency 


Ver bal/Bechanical 


3.  Future  Plans 

K.  What  hardware  do  you  expect  to  be  replaced  in  the  next 
five  (5)  years?  Indicate  approximate  replacement  date 
and  new  system  to  be  installed. _ 


B.  Ire  there  any  new  areas  of  support  which  you  believe 
will  be  important  in  the  next  five  (5)  years? _  _ 
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C.  what  is  the  aost  iaportant  problea  now  facing  your 
organization? _ _ _ _ _ _ _ 


D.  What  do  poo  believe  is  the  solution  to  increasing  coaputer- 
enhancad  productivity  at  Dili? _ 


U.  Current  Projects 

I.  ire  there  any  plans  in  progress  or  planned  which  would  have  a 
bearing  on  this  study? _ _ _ _ _ _ _ 


B.  k re  there  existing  or  planned  DIU  operational  capabilities 

requiring  additional  analysis?_ _ 


C.  Coaaents 
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PART  III:  DIRECTION /POLICIES 

A.  How  are  work  assignaents  made  in  your  organization? 


B.  Hhat  are  the  long-tera  objectives  of  your  organization? _ 


C.  Describe  your  overall  plan  to  identify,  evaluate  and  introduce 
new  tools  and  techniques  to  DBA? _ 


Hhat  percent  of  your  operational  software  was  orginally 
developed  by  a  subcontractor? _ 

Hhat  percent  of  your  software  is  aaintained  by 
Subcontractor? _  DBA  personnel? _ 

Hhat  percent  of  new  software  developaent  will  be  done  by 
Subcontractor? _  DBA  personnel?__ _ 

is  there  a  general  policy  regarding  these  allocations? 


Hhat  are  your  long  range  plans  for  in-house  training  for  aoder 
prograeaing  environment  practices? _ 

How  ate  you  future  software  data  processing  reguireaents 
defined?_  _ _ _ 


Hhat  levels  of  technical  and  aanageaent  personnel  are  involved 
in  f u t e  reguireaents  planning? _ 
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H.  What  reports  are  received  on  pro ject  (s) ,  how  often  are  they 
received  and  are  they  verbal  or  aechanical? 

Report  Frequency  Ver bal/Hechanical 


./. 


I.  Describe  any  current  and/or  projected  capabilities/deficiences 
at  DMA  as  they  relate  to  adequately  perforeinq  required 
proqranaing  functions.  _ 


Describe  any  existing  or  planned  DRi  operational  capabilities 
requiring  additional  analysis?  _ 
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PART  IV:  TOOLS/COH RENTS 


A.  Check  each  area  in  which  you  use  oc  could  use  a  software  tool 
to  increase  your  productivity. 

(u  «  currently  used;  c  *  could  use). 

REQUIREMENTS 

_  Definition 

_  Validation 

DESIGN 

_  Simulator 

Prograa  design  language 
_  Standardization 

DEVELOPMENT 

_  Coapiler 

_  MAP  processor 

Assenbler 

_  Linkage  editor 

Text  editor 

_  Configuration  control 

Security 

TESTING 

_  Test  generator 

Test  validator 

DOCUMENTATION 

_  Flowchart  generator 

_  Automated  text  aanageaent  systea 

_  Software  docuaentation  language 

Graphics  aids 

MANAGEMENT 

_  Cost  estiaating 

Budget  tracking 

_  Report  generator 

_  Historical  data  base 

OTHER 


B.  Do  the  tools/techniques  in  DMA  organizations  interface 

_ Nell  With  effort  _ _ Poorly. 

C.  Would  a  new  tool  be  easily  introduced  into  your  DMA 

organization. _  If  not,  please  explain  why. _ 
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Are  there  any  general  comments  you  would  like  to 
■alee  concerning  this  questionnaire?  Please  indicate 
any  area  you  believe  we  did  not  cover. 
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APPENDIX  B 

DEMONSTRATION  RESPONSE  FORM 
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MS/DBA  TOOL  DIBOBSTBATXOI  IfilOATIOB 

1.  hi*  of  tool  being  evalaatad:  ______________ __________________ 

2.  Toor  aaoo:  _ _ _ _ 

3.  Toor  organization:  _______________________________________________ 

а.  Tear  coaaereial  telophooe  aunber:  _____________________________ 

S.  Toor  evaluation  of  tk«  oasa  of  input  data  praparatioa: 

_____  High,  _____  Bediua,  ___  Low 

б.  Are  there  aodif ieatioas  to  tha  inpat  data  praparatioa  that  would 
■aka  tha  tool  aaaiar  to  oaa  la  tha  DBA  environment? 

_____  Tea,  _____  Bo 

7.  If  yea,  daacriba  tfaaaa  aodif icatlona: 


B.  Toor  evaluation  of  tha  aaaa  of  naderataadiag  tha  output  raaulta: 
_____  High,  .  Badiua,  _____  low 

9.  Ara  thara  aodifieatioao  to  tha  output  raaulta  foraat  that  would  aaka 
tha  tool  aora  uaaful  ia  tha  DBA  environment?  _____  lea,  _____  Bo 
TO-  If  yaa,  daacriba  tbaaa  aodif icationa:  ______________________________ 


11.  Do  you  perceive  aa  application  of  tha  tool  to  DBA  projects  ia  tha 

aear~tera  (TT  1982)?  _____  Taa,  _  Bo 

12.  If  yaa,  which  particular  projects  sad  how  would  you  apply  tha  tool 
to  aach  project? 

tliJlSl  APPlidtiOB 

a.  - .  a.  ____________________________________ 

b.  _  b. _ 
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1.  So  yon  perceive  as  application  of  this  tool  to  Oil  projects  la  tha 

far  tora  CM  UBS)?  .  Tea,  _ _  _  Bo 

1*.  Sf  70s,  which  particalar  projects  sad  bos  would  70a  apply  the  tool 
to  each  pro jact 7 

Project  lpcllcatioa 


a. 

b. 

c. 


a. 

b. 

c. 


4. 


d. 


IS.  poos  this  tool  base  functions  that  are  also  present  in  cnrrontly 
available  OBI  tools?  _____  Tea,  .  Bo 
It.  If  yea.  abet  are  these  functions  and  in  which  currently  available 
tool? 

tl&Slififl  EMIinill_i2iilifeIi_Ia2l 

a.  _  a. _ _ 

b.  _ b. _ 

c.  .  _  c.  _____________________________________ 

d.  _ _ _ d.  _ 

17.  Xi  both  the  deaonstrated  tool  and  the  cnrrently  available  tool  have 

the  sane  function,  which  one  would  you  prefer  to  use  and  why 
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EVALOATION  TOPI,  gET 

The  following  set  of  tools  was  selected  by  the  process  ex¬ 
plained  in  section  3.1. 

FAME  :  Front-End  Analysis  6  Modeling  Environment 

FAME  is  a  microprocessor  based  system  for  interactively 
developing,  analyzing  and  displaying  Higher  Order  Software 
(HOS)  and  other  system  models  in  a  user  friendly  environment. 
The  nature  of  the  HOS  model  is  such,  that  when  completed  it 
can  be  the  basis  for  projection  to  a  variety  of  forms  such  as 
Structured  Design  Diagrams,  SADT  and  IDEF  Diagrams,  Petri- 
Nets,  Data  Flow  Diagrams,  PSL/PSA  Source  Code, etc.  The 
user's  interface  with  the  analyzer  is  easily  recognized  by 
any  current  user  of  a  structured  modeling  approach;  therefore 
extensive  training  is  unnecessary.  Futhermore,  when  all  the 
system  capabilities  are  used  one  can  check  on  proper  usage  of 
data  types,  functions  and  control  structures  and  thereby  add 
a  new  dimension  to  the  design  process  that  will  lead  to 
better,  and  more  easily  verified  software  designs.  system 
features  include:  prompted  interactive  model  development; 

analysis  of  modeling  errors;  graphic  output  of  models;  and 
conversion  programs  to  a  nuiiier  of  standard  methodologies. 

FORMAT 

FORMAT  is  a  text  processor  which  is  a  useful  tool  for  anyone 
involved  in  producing  documentation,  reports,  correspondence, 
or  other  written  material.  A  text  processor  automatically 
does  many  of  the  tedious  and  time  consuming  chores  needed  to 
produce  a  finished  product,  such  as  right  margin 
justification,  page  numbering,  chapter  and  section  numbering, 
centering,  table  of  contents  and  index  generation,  and  other 
similar  operations. 

IS/1  WORKBENCH  FOR  VAX 

The  IS/1  Workbench  for  the  VAX  is  a  facility  that  provides  a 
convenient  working  environment  and  a  uniform  set  of  tools  for 
computer  program  development,  document  preparation  and  text 
processing.  It  is  a  general-purpose,  multi-user,  interactive 
system  based  on  Bell  Laboratories'  PWB/UNIX  system 
specifically  engineered  to  make  the  designer's,  programmer's 
and  documenter's  environment  simple,  efficient,  flexible  and 
productive. 

MAPPER 
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MAPPER  is  a  real-time  data  base  management  system  in  tbe 
UNI V AC  series  1100  environment.  The  software  system  is 
specifically  designed  to  efficiently  support  the  intense  mix 
of  activity  inherent  in  the  Real-Time  Report 
Processing/Generating  environment  and  still  allow  demand  and 
batch  background  processing.  The  Series  1100  Operating 
System  interfaces  with  MAPPER  1100  functions  through  the 
MAPPER  Supervisor,  which  controls  terminal  polling,  function 
loading  and  execution,  and  storage.  Breakpoint  and  usage  al¬ 
gorithms  are  established  by  which  MAPPER  Supervisor 
prioritizes  all  internal  activity  to  minimize  response  time, 
giving  highest  priority  to  low  impact  activities. 

NODAL 

NODAL  is  an  execution  path  flow  analyzer  designed  to  aid  the 
user  in  executing  all  the  source  code  and  all  the  branches  in 
testing  a  FORTRAN  program.  It  uses  the  technique  of 
analyzing  the  code  that  will  record  the  execution  of  the 
program's  nodes.  At  the  normal  end  of  an  execution  of  the 
user's  instrumented  program,  NODAL  will  obtain  control  and 
provide  information  about  the  frequency  of  execution  of  each 
node.  Also  provided  is  a  test  effectiveness  ratio  (nodes 
executed/nodes  identified)  for  each  routine,  a  test  effec¬ 
tiveness  ratio  for  the  entire  program,  and  a  list  of  the  pro¬ 
gram  nodes  not  executed. 

OPTIMA 

The  SPERRY  UNIVAC  1100  Project  Management  System  (OPTIMA 
1100),  an  integrated  system  for  project  planning  and  control, 
is  based  on  networking  techniques.  The  OPTIMA  1100  System 
performs  time  analysis,  cost  analysis,  resource  analysis, 
resource  allocation,  report  processing,  including  network 
plots,  and  maintenance/updating  of  OPTIMA  1100  mass  storage 
files.  The  overall  design  is  an  integrated  system  comprising 
these  functions. 

SCMS  :  Software  Complexity  Measurement  System 

SCMS  is  an  analysis  tool  that  computes  three  types  of  com¬ 
plexity  (Cyclomatic,  Essential,  and  Actual)  and  graphically 
displays  the  control  structure  for  each  module  of  an  input 
program.  The  complexity  measure  is  based  on  a  graph- 
theoretic  approach  to  the  analysis  of  programs  developed  by 
McCabe  and  provides  information  about  how  complicated 
(Cyclomatic  Complexity) ,  how  well  structured  (Essential 
Complexity) ,  and  how  well  tested  (Actual  Complexity)  a  module 
is.  This  tool  also  provides  a  tree  data  structure  that  will 
identify  modular  interaction  for  the  entire  program. 
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SDDL  :  Software  Design  and  Documentation  Language 


The  objective  of  the  Software  Design  and  Documentation 
Language  (SDDL)  is  to  provide  an  effective  communications 
medium  to  support  the  design  and  documentation  of  complex 
software  applications.  This  objective  is  met  by  providing 

(1)  a  processor  which  can  convert  design  specifications  into 
an  intelligible,  informative  machine-reproducible  document, 

(2)  a  design  and  documentation  language  with  forms  and  syntax 
that  are  simple,  unrestrictive,  and  communicative,  and  (3) 
methodology  for  effective  use  of  the  language  and  processor. 
The  processor  has  the  capability  to  format  documents,  sum¬ 
marize  design  information  in  the  form  of  reports  and  handle 
various  user-controlled  directives. 


The  TX  text  editor  is  a  stand-alone  interactive  program  that 
is  intended  exclusively  for  full-screen,  interactive  text 
editing  of  ASCII  files  on  Harris  Models  2300,  8610,  and  8680 
CRT's. 


APPENDIX  D 

LIFE  CYCLE  QUESTIONNAIRES 
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REQUIREMENTS  SPECIFICATION  EVALUATION  CRITERIA 

1.  Are  you  familiar  with  any  of  tha  following  require- 
otnti  specification  techniques? 

-  PSL/PSA  (University  of  Michigan) 

-  Structured  Analysis  (Yourdan) 

-  ISDOS  (University  of  Michigan) 

-  CADSAT  (U.  S.  Air  rorce) 

-  SREP  (U.  S.  Army) 

-  SADT  (SofTech) 

-  SAMM  (Boeing) 

-  RLP  (GTE) 

-  SREM  (TRW) 

-  IDEr  (SofTech) 

-  If  yes,  is  your  knowledge  from 

general  background  _ 

formal  training  _ 

actual  use 


2.  Is  the  requirements  tool  user  friendly  (that  is)? 
Is  it  easy  to  learn? 

Is  it  easy  to  use? 

Does  it  promote  user  satisfaction? 

-  Are  error  diagnostics  understandable 
without  recourse  to  study  or  documentation? 
Does  it  provide  help  facilities? 

-  Does  it  recognize  different  levels  of  users? 
(That  is  -  from  novice  to  experienced)  , 

If  yes,  characterize  the  levels  as  you 
perceive  them. 


3.  Does  the  tool  ease  the  task  of  decomposing 
the  problem  into  functions? 

4.  Does  the  tool  allow  traceability  between 
requirements  and  design? 

5.  Does  the  tool  identify  inconsistencies 
in  requirements? 

6.  Does  the  tool  promote  a  top-down  approach? 

7.  Is  user  documentation  task  reduced  through 
the  use  of  the  tool? 
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8.  a.  Is  productivity  lncrsasad  through  usa  of  tha  tool? 

(That  la  -  Is  thara  a  reduction  In  arrors 
attrlbutad  to  improved  requirements 
s pacification.) 

b.  Wara  bo r a  spaclfic  requirements  surfaead  as  a 
rasult  of  using  this  tool? 

9.  How  much  tins  was  spant  In  training 
(formal  and  on-the-job)? 

FORMAL  OJT 

-  1  to  2  hrs  _  _ 

-  2  to  5  hrs  _  _ 

-  5  to  10  hrs  _  _ 

-  graatar  than  10  hrs  _  _ 

How  touch  tima  is  appropriata  for  othar  DMA 
usars  to  laarn  to  usa  this  tool? 

(Exprass  in  terms  of,  work  days.) 


10.  Is  tha  lavel  of  datall  for  the  problem 
statement  sufficient  to  allow  requirements 
specification? 

11.  Does  the  tool  do  what  it's  advertised  to  do? 

12.  Do  you  think  this  tool  is  applicable  to  the 
DMA  environment?  Explain. 


13.  Provide  your  assessment  of  the  tool  and,  if 

possible,  compare  this  tool  with  other  require¬ 
ments  tools/techniques  with  which  you  are 
familiar. 


14.  Succinctly  describe  the  best  feature(s) 
of  this  tool. 


IS.  Succinctly  describe  the  worst  feature(s) 
of  this  tool. 
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Provide  any  comments  about  this  tool  and 
thia  portion  of  the  evaluation  which  were 
not  surfaced  by  the  foregoing  questions. 


206 


DESIGN  DEFINITION  EVALUATION  CRITERIA 


Are  you  familiar  with  any  of  tha  following  design 
tachniquas 

-  SADT 

-  SDDL 

-  JACKSON 

-  HOS 

-  HIPO 

-  WARNIER 

-  ORR 

-  PETRI-NETS 

-  If  yes,  ia  your  knowladga  from 

ganaral  background  _ 

formal  training  _ 

actual  uaa  _____ 

la  tha  DESIGN  tool  uaar  friandly  (that  la)? 

-  Is  it  aasy  to  laarn? 

-  Is  it  aasy  to  uaa? 

-  Doas  it  promota  uaar  satisfaction? 

-  Art  arror  diagnostics  understandable 
without  recourse  to  study  or  documentation? 

-  Doas  it  provide  help  facilities? 

-  Doas  it  recognize  different  levels  of  users? 
(That  is  -  from  novlca  to  experienced) 

If  yes,  characterize  the  levels  as  you 
perceive  them. 


3. 


5. 

S. 

7. 

B. 

9. 

10. 


Does  the  tool  ease  the  task  of  defining 
the  problem  functions? 

Does  the  tool  allow  traceability  between 
requirements  and  design? 

Between  design  and  coding? 

Does  the  tool  promote  modularity  in  design? 

Does  the  tool  promote  a  top-down  approach? 

Is  user  documentation  task  reduced  through 
the  use  of  the  tool? 

Was  the  coding  task  shortened  or  eased  in 
any  way  by  the  use  of  this  tool? 

a.  Is  productivity  Increased  through  use  of  the  tool? 
(That  is  -  is  there  a  reduction  in  errors 
attributed  to  Improved  design  definition. 

b.  Were  more  new  requirements  surfaced  as  a 
result  of  using  this  tool? 
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11.  How  much  elm*  mi  spent  in  training 
{ formal  and  on-the-job)? 

FORMAL  OJT 

-  1  to  2  hr  i  _  _ 

-  2  to  5  hr.«  _  _ 

-  5  to  19  hrs  _  _ 

-  graatar  than  18  hra  _  _____ 

How  much  tima  la  appropriata  for  othar  DMA 
usars  to  laarn  to  uaa  thla  tool? 

(Cxprass  in  terms  of  work  days.) 


12.  Was  the  level  of  detail  in  the  requirements 
statement  sufficient  to  allow  dasign 
definition? 

13.  Does  the  tool  do  what  it's  advertised  to  do? 

14.  Do  you  think  this  tool  is  applicable  to  the 
DMA  environment?  Explain. 


15.  Provide  your  assessment  of  the  tool  and,  if 

possible,  compare  this  tool  with  other  design 
tools/techniques  with  which  you  are 
fam il iar . 


15.  Succinctly  describe  the  best  feature(s) 
of  this  tool. 


17.  Succinctly  describe  the  worst  feature(s) 
of  this  tool. 
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IS.  Provide  any  eoaaenta  about  this  tool  and 
thla  portion  of  the  evaluation  which  were 
not  aurfaced  by  the  foregoing  queatlona. 
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CODING  PHASE  EVALUATION  CRITERIA 


TOOL  EVALUATED; 


1.  Are  you  familiar  with  any  of  tha  following  coding 
tachniquaa  or  practices? 

-  FORTRAN  Structured  Programming  Concepts 

-  Top-down  Implementation 

-  Structured  FORTRAN  preprocessors 

-  Modularization  Criteria 

-  COMMON  Data  Usage  Guidelines 

-  Formal/Actual  Subroutine  Parameter 
Conventions 

-  Program  Documentation  Critarla 

-  Code  Identification  Guidelines 

-  If  yes,  is  your  knowledge  from 

general  background  _ 

formal  training  _ 

actual  use  _ 


2.  Is  the  coding  tool  user  friendly  (that  is)? 

Is  it  easy  to  learn? 

Is  it  easy  to  use? 

-  Does  it  promote  user  satisfaction? 

-  Are  error  diagnostics  understandable 
without  recourse  to  study  or  documentation? 

-  Does  it  provide  help  facilities? 

-  Does  it  recognize  different  levels  of  users? 
(That  is  -  from  novice  to  experienced) 

If  yes,  characterize  the  levels  as  you 
perceive  them. 


3.  Does  the  tool  ease  the  task  of  coding 
the  designed  functions? 

4.  Does  the  tool  allow  traceability  between 
design  and  coding? 

5.  Does  the  tool  allow  traceability  between 
coding  and  testing? 

5.  Does  the  tool  promote  modularity  in  coding? 

7.  Does  the  tool  promote  a  top-down  approach? 

8.  Is  the  user  documentation  task  reduced  through 
the  use  of  the  tool? 

9.  Was  the  coding  task  shortened  or  eased  in 
any  way  by  the  use  of  this  tool? 
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YES  WO 


1>.  a.  la  productivity  lncraasad  through  uaa  of  tha  tool? 
(That  la  -  la  thara  a  reduction  In  arrora 
attributed  to  improved  coding  capabil Itlee?) 

b.  Were  design  modifications  identified  as  a  result 
of  using  this  tool? 

11.  How  such  tine  was  spent  In  training 
(formal  and  on-the-job)? 

rOBHAL  OJT 

-  1  to  2  hra  _  _ _ 

-  2  to  5  hra  _  _ 

-  5  to  lfl  hra  _  _____ 

-  greater  than  lfl  hrs  _  _ 

How  much  tine  Is  appropriate  for  other  OWA 
users  to  learn  to  use  this  tool? 

(Express  in  terms  of  work  days.) 


12.  Was  tha  level  of  detail  In  the  design 
statement  sufficient  to  allow  code 
Implementation? 

13.  Does  the  tool  do  what  it's  advertised  to  do? 

14.  Do  you  think  this  tool  is  applicable  to  the 
DMA  environment?  Explain. 


15.  Provide  your  assessment  of  the  tool  and,  if 
possible,  compare  this  tool  with  other  coding 
tools/techniques  with  which  you  are 
familiar. 


15.  Succinctly  describe  the  best  feature(s) 
of  this  tool. 


17.  Succinctly  describe  the  worst  feature(s) 
of  this  tool. 
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18.  Provide  any  consents  about  this  tool  and 
this  portion  of  the  evaluation  which  were 
not  surfaced  by  the  foregoing  questions. 


* 

1 


TESTING  PHASE  EVALUATION  CRITERIA 


TOOL  EVALUATED! 


1.  Are  you  familiar  with  any  of  tha  following  taating 
tools? 

-  FAVS 

-  NODAL 

-  SCMS 

-  RXVP93 

-  DAVE 

-  FASP 

-  FORTRAN' 77  ANALYZER 

-  SOFTOOL  80 

-  if  yes,  is  your  knowledge  from 

ganaral  background  _ 

formal  training  ___ 

actual  use 


2.  is  the  testing  tool  user  friendly  (that  is)? 

-  Is  it  easy  to  learn? 

-  Is  it  easy  to  use? 

-  Does  it  promote  user  satisfaction? 

-  Are  error  diagnostics  understandable 
without  recourse  to  study  or  documentation? 

-  Does  it  provide  help  facilities? 

-  Does  it  recognize  different  levels  of  users? 
(That  is  -  from  novice  to  experienced) 

If  yes,  characterize  the  levels  as  you 
perceive  them. 


3.  Does  the  tool  ease  the  task  of  testing 
the  designed  functions? 

4.  Does  the  tool  allow  traceability  between 
coding  and  testing? 

5.  Does  the  tool  promote  modularity  in  testing? 

6.  Is  the  user  documentation  task  reduced  through 
the  use  of  the  tool? 

7.  Was  the  testing  task  shortened  or  eased  in 
any  way  by  the  use  of  this  tool? 


i 
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S.  a.  la  productivity  increased  through  uaa  of  tha  tool? 
(That  ia  -  la  thara  a  reduction  in  errors 
attributed  to  lnproved  tasting  capabilities?) 


YES  >10 


b.  Ware  design  modifications  identified  as  a  result 
of  using  this  tool? 

9.  Dow  much  time  was  spent  in  training 
(formal  and  on-the-job)? 

FORMAL  OJT 

-  1  to  2  hrs  _  _ 

-  2  to  5  hrs  _  _ 

-  5  to  19  hrs  _  _ 

-  greater  than  10  hrs  _____  _ 

How  much  time  Is  appropriate  for  other  DMA 
users  to  learn  to  use  this  tool? 

(Express  in  terms  of  work  days.) 


10.  Was  the  level  of  detail  in  the  design 
statement  sufficient  to  allow  comprehensive 
testing? 

11.  Does  the  tool  do  what  it's  advertised  to  do? 

12.  Do  you  think  this  tool  is  applicable  to  the 
DMA  environment?  Explain. 


13.  Provide  your  assessment  of  the  tool  and.  If 
possible,  compare  this  tool  with  other  coding 
tools/ techniques  with  which  you  are 
familiar. 


14.  Succinctly  describe  the  best  feature(s) 
of  this  tool. 


IS.  Succinctly  describe  the  worst  feature(s) 
of  this  tool. 
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Provide  any  eoninti  about  thla  tool  and 
thla  portion  of  tha  evaluation  which  were 
not  aur faced  by  the  foregoing  queatlona. 


APPENDIX  E 

EVALUATION  SURVEY  RESPONSES  SUMMARIZED 


DM A_TBAM_ PARTICIPANTS 


DMAAC 

Geopositional  Department 
Photogrammetric  Control  Division 

Aerospace  Cartography  Department 
Cartographic  Data  Division 

Scientific  Data  Department 
Scientific  Computer  Division 


DMAHTC 

Geodesy  Department 
Satellite  Geophysics  Division 

Topography  Department 
Techniques  S  Programming  Division 

Computer  Services  Department 
Scientific  Computing  Division 


Computer  Services  Department 
Techniques  Office 


Robert  Spors 


Norbert  Pink 


Larry  Holmgren 
R.  Dvane  Kindsfather 
Charles  Masback 
Billy  Rice 


Peter  Mayer 


Johnnie  Bishop 


Mary  Albert 
Mike  Levis 
Geri  Loughney 

Martha  Plemmons 
Thomas  P.  Williams 
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1)  In  the  following  the  nuaeric  data  is  represented  as  AC/HTC. 

2)  kabiguous  answers  such  as  "fairly",  "soaewhat",  and  "so-so" 
were  annotated  as  a  "yes"  a  "no". 

3)  Sentences  aay  be  paraphrased  to  express  aain  content. 

4)  If  ranges  were  given,  upper  liait  was  used. 

5)  Coaaents  are  prefaced  with  "HTC"  or  "kC". 

RgpOj REHjNTS  $ pEC| F IC ATIOM  BV k LUATIQN . CBITE8 Ik 
7b0TH  CENTERS  Usi5  P»HE) 

1.  kre  you  faailiar  with  any  of  the  following  require 
aents  specification  techniques? 

PSL/PSA  (University  of  Michigan) 

Structured  Analysis  (Yourdan) 

ISDOS  (University  of  Michigan) 

-  CADSAT  (U.  S.  Air  Force) 

-  SREP  (0.  S.  Aray) 

-  SADT  (SofTech) 

-  SAMM  (Boeing) 

-  BLP  (GTE) 

-  SREM  (TRW) 

-  IDEF  (SofTech) 

If  yes,  is  your  knowledge  froa 

general  background  _1Z1_ 
foraal  training  -111- 

actual  use  -21Q_ 

2.  Is  the  requireaents  tool  user  friendly  (that  is)? 

Is  it  easy  to  learn?  4/4  -HZ- 

HTC  -  provided  good  quality  foraal  training  available. 

Is  it  easy  to  use? 

Does  it  proaote  user  satisfaction? 

Are  error  diagnostics  understandable 
without  recourse  to  study  or  docuaentation? 

Does  it  provide  help  facilities? 

HTC  -  help  facilities  aren’t  helpful. 

Does  it  recognize  different  levels  of  users?  -kll- 

(That  is  -  froa  novice  to  experienced) 

If  yes,  characterize  the  levels  as  you 
perceive  thea. 

3.  Does  the  tool  ease  the  task  of  decoaposing 

the  problea  into  functions?  -HQ- 

«.  Does  the  tool  allow  traceability  between 


-HL-  -Ul- 
_Ul. 

0^1  f>'\ 

.3/5. 


_II§ _ SQ~ 

-U'—  _!i^_ 

-HI-  -iil- 
_2^2_  Alk. 

2<i2_  6^6 

_Lz:2_  Alk. 
-1 IL.  All- 
-QlQ—  Alk- 
0/0  _6^6_ 
0^0_  6^6. 

All-  .5^6. 
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requirements  and  design? 


JL ZL.  -Ul- 


6. 

7. 

8. 


9. 


Does  the  tool  identify  inconsistencies 

in  requireaents?  5/6  _2Z2. 

Does  the  tool  proaote  a  top-down  approach?  5/6 

Is  user  docuaentation  task  reduced  through 

the  use  of  the  tool?  tt/tt 

a.  Is  productivity  increased  through  use  of  the  tool?  ~U/3 
(That  is  -  is  there  a  reduction  in  errors 
attributed  to  iaproved  requireaents 
specification. ) 

b.  Were  aore  specific  requireaents  surfaced  as  a 

result  of  using  this  tool?  3/w  _3^1, 


_2Z1_ 

.m. 


How  nuch  tiae  was  spent  in  training 
(foraal  and  on-the-job)? 


-  1  to  2  hrs 

-  2  to  5  hrs 

-  5  to  10  hrs 

-  greater  than  10  hrs 


-umi- 

_2zi_ 

-2Z2_ 

_IZ2_ 

_2Z2_ 


„B2I— 

_2Zi_ 

_flzi_ 

_2Z2_ 

-1Z2- 


How  auch  tiae  is  appropriate  for  other  DBA 
users  to  learn  to  use  this  tool? 

(Express  in  teras  of  work  days.) 


AC  -  2,  5,  1,  2.  1 

AC  -  Does  not  fit  current  procedures  or  policies; 
requires  decisions  by  Sr.  Analysts  that  they 
are  not  in  a  position  to  sake. 

HTC  -  10,  10,  10,  1,  3 


10. 

Is  the  level  of  detail  for  the  problea 
stateaent  sufficient  to  allow  requireaents 

specification? 

_2Zfc_  _«Z°_ 

11. 

Does  the  tool  do 

what  it's  advertised  to  do? 

_2Z!i _ 2Z2_ 

12. 

Do  you  think  this 

tool  is  applicable  to  the 

DBA  environaent? 

Explain. 

-2ZL.  _2Zi_ 

AC  -  Could  be  if  context  and  use  were  well  thought  out 
AC  -  Yes,  forces  top-down  approach 
AC  -  Ho 

AC  -  Requires  designer  to  think  through  specification 
HTC  -  There  is  a  need  for  better  docuaentation  and  aore 
detailed  initial  requireaents 
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HTC  -  Pew  people  are  requested  to  handle  a  project  f roe 
beginning  to  end. 

HTC  -  would  help  define  all  requirements  of  a  prograe  in  the 
beginning  eliminating  wasted  tine  in  recoding  problea. 
HTC  -  Extensive  aaount  of  software  and  rapid  prograaaer 
turnover  aakes  this  tool  useful  to  DMA 
HTC  -  No,  too  many  bugs. 

13.  Provide  your  assessaent  of  the  tool  and,  if 
possible,  coa pare  this  tool  with  other  require- 
aents  tools/techniques  with  which  you  are 
faailiar . 

AC  -  Requires  too  auch  data  and  information  from  user. 

AC  -  Good  and  useful. 

AC  -  Poor. 

AC  -  Gets  you  started  with  an  overall  view;  organizes 
the  problea. 

HTC  -  Good  tool;  would  be  helpful  in  aaintenance;  high 
quality  foraal  training  needed. 

HTC  -  Unnecessary. 

HTC  -  Very  good;  reduces  tine  to  find  solution;  eliainates 
wasted  prograaaing  tiae;  shortens  tine  to  get  prograa 
into  production. 

HTC  -  Has  possibilities  but  too  new,  i.e.,  bugs. 

14.  Succinctly  describe  the  best  feature  (s) 
of  this  tool. 

AC  -  Computer  produced  tree. 

AC  -  Graphic  tree  diagram;  paraaeter  checking;  paront- 
offspring  diagram;  excellent  documentation  produced. 

T. C  -  Error  checking;  documentation  of  flow  processes 
HTC  -  Output  is  good  docuaentation;  tree  diagram  is  useful 
in  coding;  prompts  are  helpful;  fairly  easy  to  use 
once  learned. 

HTC  -  Analysis. 

HTC  -  Top-down  functional  decoaposition. 

HTC  -  Analysis;  aid  in  organization  and  logic;  good 
documentation  for  aaintenance;  aids  in  problea 
solution;  catches  errors  early. 

HTC  -  Tree  diagram. 

15.  Succinctly  describe  the  worst  feature  (s) 
of  this  tool. 

AC  -  Requires  too  auch  data. 

AC  -  Sometimes  very  complex. 

AC  -  Error  messages  are  cryptic. 

AC  -  User  documentation. 
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BTC  -  Errors  bard  to  correct;  aany  bugs;  only  allows 
single-step  update. 

HTC  -  Bugs  in  software. 

BTC  -  Errors  hard  to  correct;  bad  error  eessages; 

problae  with  global  update;  liaited  nueber  of  variables. 

16.  provide  any  coaaents  about  this  tool  and 
this  portion  of  the  evaluation  which  were 
not  surfaced  by  the  foregoing  questions. 

1C  -  Bajor  problea  faced  was  bad  data  coaaunications  line 
requiring  repetitive  work  by  teas  aeabers. 

1C  -  The  syntax  of  the  tool  could  distract  froa  the 
original  requireaents. 

1C  -  Could  be  used  in  design,  but  inappropriate  for 
requireaents. 

BTC  -  Poorly  docuaented;  for  different  _2Bti2aS_*  different 
_S.iEa£_  of  answers  were  required,  i.e.,  nuaeric  or 
alphanuaeric  and  soaetiaes  it  was  irrelevant;  learning 
tool  would  require  auch  use;  tool  is  redundant  co  work 
accoaplished;  aeaningless  error  aessages  given;  correcting 
errors  is  difficult. 

HTC  -  Best  feature  is  analysis;  diagnostics  _B2t-  aset 

friendly;  only  allows  one  update  of  a  specific  group  at  a 
ties;  aultiple  errors  in  docuaentation;  1-J  terainal  has 
nice  qualities  but  difficult  to  use. 

HTC  -  Poor  docuaentation. 

HTC  -  Project  too  siaple;  errors  in  Paae  software; 
poor  user's  aanual. 


(1C  USED  SDDL,  HTC  USED  F1HE) 


Are  you  faailiar  with  any  of  the  following  design 
techniques 

_ 62 _ 

-  SADT 

_£L2l_ 

.625 

-  SDDL 

_i2i_ 

.525. 

-  JACKSON 

.US. 

.626. 

-  HOS 

.ICS. 

.526. 

-  HIPO 

-ICi- 

4^4 

-  WABNIEB 

_i2I_ 

"AcV. 

-  OBH 

so 

.625. 

-  PETRI-NETS 

_i2i_ 

.526. 
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-  If  yes,  is  your  knowledge  from 
general  background  2/1 

foreal  training  ~p/1 

actual  use  0/p 

2.  Is  the  DSSIGR  tool  user  friendly  (that  is)  ? 


-  Is  it  easy  to  learn? 

4/4 

-U2- 

-  Is  it  easy  to  use? 

JUL. 

-11 1- 

Does  it  proaote  user  satisfaction? 

-  ire  error  diagnostics  understandable 

_2 n. 

-in. 

without  recourse  to  study  or  docuaentation? 

1/1 

-5/5- 

Does  it  provide  help  facilities? 

Z215- 

-•Ul. 

Does  it  recognize  different  levels  of  users? 
(That  is  -  froa  novice  to  experienced) 

If  yes,  characterize  the  levels  as  you 
perceive  thea. 

-111- 

_&/«_ 

3. 

Does  the  tool  ease  the  task  of  defining 
the  problea  functions? 

2Zi_ 

-1/2- 

4. 

Does  the  tool  allow  traceability  between 
reguireaents  and  design? 

_4/5_ 

-2/2- 

5. 

Between  design  and  coding? 

-115- 

-1/1- 

6. 

Does  the  tool  proaote  nodularity  in  design? 

JULt- 

-2/2, 

7. 

Does  the  tool  proaote  a  top-down  approach? 

-5/6- 

-1/1- 

8. 

Is  user  docuaentation  task  reduced  through 
the  use  of  the  tool? 

-H&- 

-2/2- 

9. 

Was  the  coding  task  shortened  or  eased  in 
any  way  by  the  use  of  this  tool? 

-215- 

-«Z1. 

10. 

a.  Is  productivity  increased  through  use  of  the  tool? 

-5HL- 

-4/1. 

(That  is  -  is  there  a  reduction  in  errors 
attributed  to  iaproved  design  definition. 


b.  Here  sore  new  requirements  surfaced  as  a 

result  of  using  this  tool?  3/5  -2dl 

11.  How  auch  tiee  was  spent  in  training 
(foraal  and  on-the-job) 7 

-EQEfl&i _ 221— 

-  1  to  2  hrs  5/2  1/p 
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-  2  to  5  hrs  _i^2_  1/2 

-  5  to  10  hrs  ZltlZ 

-  greater  than  10  hrs  2/1 

How  auch  tiae  is  appropriate  for  other  DBA 
users  to  learn  to  use  this  tool? 

(Express  in  teras  of  work  days.) 

SDDL: 

1C  -  1,  3 

FAHE : 

HTC  -  1,  1,  2.  2,  3,  5 

12.  Has  the  level  of  detail  in  the  requireaents 
stateaent  sufficient  to  allow  design 
definition? 

13.  Does  the  tool  do  what  it's  advertised  to  do? 

14.  Do  you  think  this  tool  is  applicable  to  the 
DBA  environaent?  Explain. 

SDDLs 

AC  -  Allows  design  in  pseudo  code. 

AC  -  Could  be  used  in  prograa  design  phase. 

AC  -  Seeas  like  you  are  coding  the  prograa  twice. 

PAHE: 

HTC  -  Project  is  better  understood;  provides  good 
docuaentation. 

HTC  -  Shortens  tiae  for  generating  a  prograa 
HTC  -  Yes,  but  each  programmer  will  have  his  own  interface 
HTC  -  Too  aany  bugs  still  exist. 

15.  Provide  your  assessaent  of  the  tool  and,  if 
possible,  compare  this  tool  with  other  design 
tools/technigues  with  which  you  are 
faailiar . 

SDDL: 

AC  -  Unwieldly  to  use. 

AC  -  Of  little  value. 

AC  -  Good  to  aid  in  aaintenance 
AC  -  Largely  a  pretty-printer. 

AC  -  No,  DMA  does  not  work  in  an  interactive  environaent. 

rAHE: 

HTC  -  Fairly  good,  but  eaphasis  should  be  on  aetbodology 
rather  than  interaction  with  coaputer. 

HTC  -  Liaitations  of  tool  creates  problens  in  decoaposition. 
BTC  -  Too  iaaature  at  this  tiae. 

16.  Succinctly  describe  the  best  feature  (s) 
of  this  tool. 

SDDL; 


_2Z5_  Jia. 
_2^!_ 


223 


JL-  "t 


AC  -  Formatted  source  listing;  allows  definition  of  control 
structures 

iC  -  Module  invocation  tree;  cross  reference  tables. 

1C  -  Bequires  analyst  to  discipline  his  thinking. 

AC  -  It  does  cross  checking  that  a  huaan  author  Mould  not  do. 

FAME: 

HTC  -  capability  to  define  I/O. 

HTC  -  Output  is  good  docuaentation,  tree  diagraa  is  useful 
in  creation  of  design  and  coding;  proapts  helpful; 
fairly  easy  to  use  once  learned. 

17.  Succinctly  describe  the  uorst  feature (s) 
of  this  tool. 

SDDL: 

AC  -  No  paraseter  checking 

AC  -  Flags  required  for  cross  references 

AC  -  Too  inflexible 

AC  -  Doesn't  do  anything  but  a  little  editing 

AC  -  Aaount  of  inforaation  required  for  tool  to  perfora. 

FAME; 

HTC  -  Typos  not  easy  to  correct  and  aay  cause  errors  which 
are  ucaccessible;  unneeded  proapts  given;  limited 
nuaber  of  inputs;  no  convenient  way  to  stop  analysis; 
bug  in  global  update  requests;  error  Messages  weren't 
easily  understood. 

HTC  -  Expansion  liaitations;  analysis  liaitations; 
variable  liaitations. 

HTC  -  Liaitations:  size  and  nuaber;  errors  hard  to  correct; 
couldn't  access  soae  inputs;  probleas  with  analysis. 

18.  Provide  any  coaaents  about  this  tool  and 
this  portion  of  the  evaluation  which  were 
not  surfaced  by  the  foregoing  questions. 

SDDL: 

AC  -  Would  not  be  used  at  DMA  without  aanageaent  pressure. 

FARE: 

HTC  -  A  lot  of  tiae  should  be  spent  on  fornal  training; 
aultiple  probleas  should  be  solved  in  training. 

HTC  -  More  troubles  due  to  greater  detail  required. 


_£fiEIfi£_mSE_EIAimieS_EBJXEfiIA 

(AC  USED  TX,  HTC  USED  IS/1) 


Are  you  faailiar  with  any  of  the  following  coding 
techniques  or  practices? 

_I£§— 

— S2_ 

FORTRAN  Structured  Prograaaing  Concepts 

-kil— 

Ail 

Top-down  lapleaentation 

Ail 

Structured  FORTRAN  preprocessors 

-UZ— 

AiH 

Modularization  Criteria 

JUA— 

ah 

COMMON  Data  Osage  Guidelines 

Ail 

* 


224 


-  Forsal/hctual  Subroutine  Paraeeter 
Conventions 

Prograa  Docuaen tation  Criterin 
Code  Identification  Guidelines 

-  If  yes,  is  your  knowledge  froe 

general  background  >/>  . 

foreal  training  -111 _ 

actual  use  All _ 

2.  Is  the  coding  tool  user  friendly  (that  is) ? 

Is  it  easy  to  learn? 

Is  it  easy  to  use? 

Does  it  proaote  user  satisfaction? 

Ire  error  diagnostics  understandable 
without  recourse  to  study  or  docunentation? 
Does  it  provide  help  facilities? 

Does  it  recognize  different  levels  of  users? 
(That  is  -  froa  novice  to  experienced) 

If  yes,  characterize  the  levels  as  you 
perceive  then. 


All— 

-HL  AIL 
-Ill-  —ill— 


ail.  ail 

-iii—  -in. 
-<ni-  -iib. 

.212-  -III. 
AIL.  All. 
.1 12-  -Hi. 


3.  Does  the  tool  ease  the  task  of  coding 


the  designed  functions? 

-Hi-  -III- 

4. 

Does  the  tool  allow  traceability 
design  and  coding? 

between 

-111- 

AIL 

5. 

Does  the  tool  allow  traceability 
coding  and  testing? 

between 

AIL.  AlL 

6. 

Does  the  tool  proaote  nodularity 

in  coding? 

-1 IL.  AIL 

7. 

Does  the  tool  proaote  a  top-down 

approach? 

AIL  AIL 

P. 

Is  the  user  docunentation  task  reduced  through 
the  use  the  tool? 

AIL 

Aii- 

9. 

Has  the  coding  task  shortened  or 
any  way  by  the  use  of  this  tool? 

eased  in 

AIL 

-211- 

10. 

a.  Is  productivity  increased  through  use  of  the  tool? 

AIL  AIL 

(That  is  -  is  there  a  reduction  in  errors 
attributed  to  iaproved  coding  capabilities?) 


b.  Were  design  sodif ications  identified  as  a  result 

of  using  this  tool?  0/5  All, 
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11.  Hov  such  tine  was  spent  in  training 
(focaal  and  on-the-job)? 

-C&lfilL.  -221— 

-  1  to  2  hrs  4^1  1^2 

-  2  to  5  hrs  _2/2_ 

-  5  to  10  hrs  _Si22_  0/1 

-  greater  than  10  hrs  0/j  _1Z1_ 

Hov  each  tine  is  appropriate  for  other  DBA 
users  to  learn  to  use  this  tool? 

(Express  in  teras  of  work  days.) 

TX: 

AC  -  3,  5.  1,  1 

IS/1 : 

HTC  -  5,  2,  1,  2,  4 

12.  Has  the  level  of  detail  in  the  design 
stateaent  sufficient  to  allov  code 

iapleaentation?  4/6  .  _!/Q_ 

13.  Does  the  tool  do  what  it's  advertised  to  do?  3/5  _2/S_ 

14.  Do  you  think  this  tool  is  applicable  to  the 

DBA  environaent?  Explain.  _2Z2 _ 2/2- 

TX: 

AC  -  Only  if  DBA  goes  interactive. 

AC  -  No,  DBA  is  batch  oriented. 

AC  -  UNIVAC  editor  better. 

AC  -  If  iapleaented  on  DEC  eguipaent. 

IS/1 : 

HTC  -  Good  data  entry  aethod. 

HTC  -  UNIVAC  is  better. 

HTC  -  Easy  subaission  and  docuaentation  of  code. 

15.  Provide  your  assessaent  of  the  tool  and,  if 
possible,  conpare  this  tool  with  other  coding 
tools/techniques  with  which  you  are 
faailiar. 

TX: 

AC  -  A  great  deal  better  than  cards. 

AC  -  Basponse  too  slow, 

AC  -  Difficult  to  use  and  less  powerful  than  DNIVAC. 

AC  -  Powerful  and  useful;  need  special  function  keys. 

IS/1 : 

HTC  -  UNIVAC  and  IS/1  redundant. 

HTC  -  Access  is  a  problea. 

HTC  -  UNIVAC  just  as  effective  except  for  lack  of  terainals. 

HTC  -  UNIVAC  is  equal  to  or  better. 

HTC  -  Bore  faailiar  with  HD  editor  and  prefer  it  and 
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UMI?kC  editors . 


16.  Succinctly  describe  the  best  feature  (s) 
of  this  tool. 

Tl: 

1C  -  Context  editing. 

1C  -  Pull  page  capability. 

kC  -  Pull  screen  editing;  recovery  capability;  documentation. 

IS/1 ; 

HTC  -  Library  of  object  files. 

HTC  -  Text  editing  functions. 

HTC  -  Detachable  keyboard:  easy  to  learn;  advanced  editing 
systea. 

HTC  -  Easy  to  use  at  low  level. 

HTC  -  Editor  facilities;  easy  to  learn  and  use;  docuaentation 
good. 

HTC  -  Easy  to  learn  and  use. 

17.  Succinctly  describe  the  worst  feature (s) 
of  this  tool. 

TX: 

kC  -  Hot  user  friendly;  antagonistic. 
kC  -  Difficult  to  find  line  nuabers;  string  change 
difficult  to  use. 
kC  -  Meed  special  terainal. 

IS/1; 

HTC  -  Text  editor. 

HTC  -  Error  aessages  not  clear;  hardware  problems:  Line 
noise  and  printer. 

HTC  -  Inconsistent  access  to  files;  diagnostic  aessages; 
cursor  control  only  allowed  in  edit. 

18.  Provide  any  consents  about  this  tool  and 
this  portion  of  the  evaluation  which  were 
not  surfaced  by  the  foregoing  questions. 

TX: 

kC  -  Dflk  aust  be  interactive  for  it  to  be  useful.  Study 

showed  that  DHk  programing  environaent  is  in  stone-age. 
kC  -  Good  text  editors  are  important  to  coding. 

IS/1: 

HTC  -  Docuaentation  not  up-to-date;  access  sometimes  difficult; 
software  bugs  in  terainal;  error  aessages  not  friendly 
or  not  docuaented;  the  terainal  hardware  is  one  of  the 
best,  especially  the  keyboard. 

HTC  -  Mould  probably  have  liked  sore  if  aore  time  for 
training  had  been  available. 

_issu&s-mss-£VAimi2fi-SBixsm 

(MODAL) 
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Ire  you  faailiar  with  any  of  the  following  testing 
tools? 

-  MVS 

-  RODIL 

-  sens 

-  RIVP80 

-  DIVE 

-  PISP 

-  FORTRAN' 77  ANALYZER 

-  SOFTOOL  80 

If  yes,  is  youc  knowledge  ftoa 
general  background  _1Z2_ 
foraal  training  _2/l. 

actual  use  3/1 

Is  the  testing  tool  user  friendly  (that  is) 7 
Is  it  easy  to  learn? 

Is  it  easy  to  use? 

Does  it  proaote  user  satisfaction? 

Are  error  diagnostics  understandable 
without  recourse  to  study  or  docuaentation? 

-  Does  it  provide  help  facilities? 

Does  it  recognize  different  levels  of  users? 
(That  is  -  froa  novice  to  experienced) 

If  yes,  characterize  the  levels  as  you 
perceive  then. 


„i/i.  .m. 
-1/2-  -2/2. 
.2/2.  _i/fe_ 
-1/2.  .2/2. 
_2/2_  _2/&. 
_0/2_  _2/&. 
_2/2_  .m. 

_2/2_  _i/2_ 


o/o  «/o 
o/Q  _«/o_ 
_2/o_  _«/o_ 

_2Z2_  _2/2_ 
_2/2_  _i/2. 
_2/2_  _2/2_ 


Does  the  tool  ease  the  task  of  testing 

the  designed  functions?  1/1  _2/2_ 

Does  the  tool  allow  traceability  between  1/1  _2/2_ 

coding  and  testing? 


Does  the  tool  proaote  aodularity  in  testing? 


_2Z1_  -2/2. 


Is  the  user  docuaentation  task  reduced  through 

the  use  of  the  tool?  0/1  _2/2_ 

Was  the  testing  task  shortened  or  eased  in 

any  way  by  the  use  of  this  tool?  1/1  .2/2- 

a.  Is  productivity  increased  through  use  of  the  tool?  1/1  .2/2. 

(That  is  -  is  there  a  reduction  in  errors 

attributed  to  iaproved  testing  capabilities?) 

b.  Were  design  aodif ications  identified  as  a  result 
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of  using  this  tool? 


9. 


How  such  tine  was  spent  in  training 
(foraal  and  on-the-job)? 


-  ?  to  2  hrs 

-  2  to  5  hrs 

-  5  to  10  hrs 

-  greater  than  10  hrs 


.ZQfiflil. 

_222_ 

_2^2_ 

.m. 


_ 22I__ 

-222- 

_1Z2_ 

-24L 

_222_ 


How  euch  tile  is  appropriate  for  other  DHA 
users  to  learn  to  use  this  tool? 

(Express  in  tens  of  work  days.) 


AC  -  5,  2 


10. 

Has  the  level  of  detail  in  the  design 
statement  sufficient  to  allow  comprehensive 
testing? 

1 

cx 

Nj 

rs»l 

i 

11. 

Does  the  tool  do  what  it's  advertised  to  do? 

_1Z2_  _2Z2_ 

12. 

Do  you  think  this  tool  is  applicable  to  the 

DHA  environment?  Explain. 

_2^2_  _3^0_ 

HTC  -  Seels  good. 

AC  -  Hot  as  good  as  FATS. 

13. 

Provide  your  assessment  of  the  tool  and,  if 
possibla,  coapare  this  tool  with  other  coding 
tools/techniques  with  which  you  are 
faiiliar. 

AC  -  Very  limited  as  coepared  to  other  tools. 
AC  -  All  programming  in  FOHTRAH  '77,  hence  not 
HTC  -  Sounds  good. 

applicable  to  DHA. 

14. 

Succinctly  describe  the  best  feature  (s) 
of  this  tool. 

AC  -  Allows  dynamic  testing. 

15. 

Succinctly  describe  the  worst  feature  (s) 
of  this  tool. 

AC  -  Input  options  not  user  friendly. 

16. 

Provide  any  comments  about  this  tool  and 
this  portion  of  the  evaluation  which  were 
not  surfaced  by  the  foregoing  questions. 
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Dsed  as  a  aanageaent  tool  to  collect  statistical  data. 

HTC  -  Extreaely  usee  friendly. 

BTC  -  Knowing  where  to  put  SOE  is  complicated;  suaaing 
two  reports  is  complicated;  user  docuaentaticn 
could  be  reduced  through  use  of  the  tool;  OTS 
400  is  better  than  any  terminals  currently  at  DBI. 

HTC  -  Good  for  report  generating;  aissing  capability  of 
coabining  BIDS  arithmetically . 

HTC  -  Does  not  auto  update  on  exit;  keyboard  excellent. 

HTC  -  Mice  keyboard;  easy  to  use;  easy  to  learn. 

HTC  -  Keyboard  a  little  confusing;  aultiple  report 
coabinations  complicated. 
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SSgmi.SQBSMIS 

These  coaaents  *ete  received  during  discussions  held  at  the 
end  of  the  evaluations  at  each  center.  Cosaents  which 
duplicated  inforaation  in  the  prior  section  were  not 
included. 

AC  -  A  cosaent  was  aade  that  there  was  not  enough  tine  to  do 
the  evaluations.  Discussion  followed  citing  that  the  biggest 
iapact  on  schedule  and  available  tine  was  access/hardware 
oroblees.  Getting  and  aaintaining  phone  lines  to  the  reaote 
coaputer  sites  was  the  aa-jor  factor.  Probleas  were  encoun¬ 
tered  getting  a  line  out  of  DHAAC,  having  a  line/port  availa¬ 
ble  at  the  coaputer  site,  coaaunicating  with  line  noise 
present  and  physical  phone  availability.  Support  hardware 
also  affected  schedule.  The  first  terminal  delivered  to  sup¬ 
port  access  to  the  Software  Engineering  Systea  (SES)  at 
GD/DSD  in  Fort  Worth,  Texas  was  non-operable  when  received. 
A  second  terainal  delivered  did  wort,  but  the  node  of 
operation  was  slow  (300  baud)  due  to  SES  coaaunications 
capabilities.  Additionally,  to  support  the  tools  being  used, 
specifically  TX,  a  new  protocol  had  to  be  set-up  in  the  SES 
software  causing  an  additional  delay  in  access.  Wuaerous 
tines  during  the  evaluation  the  SES  was  down  for  scheduled 
aaintenance  or  broken  during  priae  time  business  hours.  A 
final  scheduling  problea  was  access  to  tools  through  only  one 
terainal.  This  reguired  the  sequencing  of  all  activities 
when  aany  could  have  been  perforaed  in  parallel. 

Other  consents  included  stateaents  that  the  statistics  to  be 
gathered  and  the  questionnaires  to  be  answered  could  have 
been  of  better  quality,  as  far  as  content  and  applicability. 
Mo  suggestions  were  given  for  iaproveaent.  A  final  discus¬ 
sion  centered  the  size  of  the  group  involved  in  the 
effort.  A  gener  .  agreeaent  was  that  the  group  size  for  this 
type  of  training  should  not  be  larger  than  seven. 

HTC  -  The  test-bed  problea  was  considered  to  entail  too  auch 
coding  in  proportion  to  other  tasks  involved  in  the  life- 
cycle  developaent.  The  terminals  were  well  liked  but  more 
tiaa  should  have  been  available  to  learn  the  intricacies  of 
their  use.  This  type  of  effort  was  considered  a  good  learn¬ 
ing  technique,  but  aore  exaaple  probleas  should  be  covered, 
rather  than  one  large  problea. 
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EVALUATION  QUESTIONNAIRE  RESPONSE  TRENDS 


Life- cycle 

Covered  by 

2U£ati2Ett*it2 _  _ 

_ &£__ 

Answers 

Yes/No 

HTC 

IQlkl 

Coaaents 

Positive/Negative 
AC  HTC  TOTAL 

Requirements 

Questions  2-8,  10-12 

52/50 

69/24 

121/74 

7/11 

10/16 

17/’7 

Design 

Questions  2-10,  12-14 

61/56 

84/28 

117/84 

7/10 

4/7 

11/17 

Coding 

Questions  2-10,  12-14 

42/46 

70/35 

112/81 

6/7 

9/10 

15/17 

Testing 

Questions  2-8,  10-12 

6/41 

6/0 

12/41 

1/4 

2/0 

3/4 

232 


APPENDIX  F 

EVALUATION  ACTIVITY  STATISTICS 


£5112 

IR1IRIRS. 

T-BED  DEVELOPMENT 

LABOR 

COMP. 

TOOL 

USAGE 

LABOR 

COMP. 

TOOL 

OSAGE 

T-BED 

I22L 

UES;£KLS 

BOUSS 

BOORS 

RUNS 

ESfiOBS 

&22E5 

BQ2SS 

SUES  ESSQ85 

ERRORS 

fame 

REQUIREMENTS 

45 

4 

7 

7 

79 

19 

9 

16 

17 

SDDL 

DESIGN 

35 

8 

12 

7 

100 

32 

27 

7 

19 

TX 

CODING 

15 

5 

3 

9 

128 

65 

111 

45 

18 

NODAL 

TESTING 

4 

0 

0 

8 

17 

3 

0 

0 

0 

RAPPER 

DATA  BASE 

8 

2 

5 

0 

5 

6 

12 

4 

0 

OPTIMA 

MANAGEMENT 

0 

0 

0 

0 

0 

0 

0 

0 

0 

FORMAT 

DOCUMENTATION 

8 

0 

0 

0 

0 

0 

0 

0 

0 

caiaic 

_ TRAINING. 

.IcBM. 

.CEVELOPMENT 

LABOR 

COMP. 

TOOL 

OSAGE 

LABOR 

COMP. 

TOOL 

USAGE 

T-BED 

TOOL 

IIEIcCXCLE 

HOURS 

HOORS 

RUNS 

SfiBQfiS 

RQURS 

B22B5 

RUNS 

ERRORS 

ERRORS 

FAME 

REQUIREMENTS 

118 

36 

45 

98 

79 

25 

41 

77 

36 

FAME 

DESIGN 

57 

21 

22 

52 

67 

21 

36 

65 

30 

IS/1 

CODING 

90 

27 

21 

47 

177 

68 

65 

58 

54 

NODAL 

TESTING 

0 

0 

0 

0 

0 

0 

0 

0 

0 

IS/1 

DOC/MAINT 

0 

0 

0 

0 

0 

0 

0 

0 

0 

MAPPER 

DOC/MAINT 

22 

J  8 

4 

10 

7 

5 

4 

10 

0 

OPTIMA 

MANAGEMENT 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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_ lamias.  . 

.DEVELOPMENT 

IzHBS 

c-gas 

IzSSSS 

U-EBBOBS 

tiilSS 

C-HBS 

Xi8!Ifi5  BiSSfiBBS 

XiEB&Q&S 

FINE 

220 

61 

74 

157 

225 

65 

96  159 

B3 

SDOL 

35 

8 

12 

7 

100 

32 

27  7 

19 

IS/1 

90 

27 

21 

47 

177 

68 

65  58 

54 

TI 

15 

5 

3 

9 

128 

65 

111  45 

18 

NODAL 

4 

0 

0 

8 

17 

3 

0  0 

0 

MAPPER 

30 

10 

9 

10 

12 

11 

16  14 

0 

OPTIHA 

0 

0 

0 

0 

0 

0 

0  0 

0 

FORMAT 

8 

0 

0 

0 

0 

0 

0  0 

0 

B&44£_i 

_Ea*Hic 

LABOR 

COMP. 

TOOL 

OSAGE 

T-BED 

aauRs 

HQUB5 

BESS 

8SSQE2 

SgRogs 

PABE 

445 

126 

170 

316 

83 

SDDL 

135 

40 

39 

14 

19 

IS/1 

267 

95 

86 

105 

54 

TX 

143 

70 

114 

54 

18 

NODAL 

21 

3 

0 

8 

0 

HAPPEN 

42 

21 

25 

24 

0 

OPTIMA 

0 

0 

0 

0 

0 

FORMAT 

8 

0 

0 

0 

0 

1061 

355 

434 

521 

174 
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APPENDIX  G 

CONCEPT  IMPLEMENTATION  EV ALOATION  MATRIX 


:c  X2?i:;s3  :::s::ncinoi . 

12  ZB/io vx  imoi:i's  . . . . . 
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ilMaiU 
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97  SOPTlUt  DZV  SLOP  BUT  TOOLS . 

99  iTiifiaoxufi  /a is zo  fizraoMxit . 

DI3IS1 

21  SXlOUtOI  /OB  NS161 . . . 

22  MOfillfl  BSSX6B  UI6816I . 

97  so  muz  arvuonxiT  tools . 

9t  sumimis  msu  bsvblofisit . 

CPDII6 

99  BOD til  SO OICI  SATA  tITBT  TBdlXQOSS  ... 

97  SOPTIAKZ  DZTZLOPIZVT  TOOLS . 
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TEST 
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M  ZIIOI  1ATZ  STAIDA IDS  . 

66  IZ90CS  ACC9CIT2I6  DATA  IIPOIT  AIOBALXKS 
•7  COflPUHZlSXVZ  TIAXIXIG  PIOGIAB . 

92  DZCtZASZ  TOIIAIOOID  TXlf  TO  BZIOTZS  ... 
5«  1AT0BAL  LAV50A6Z  OSZI/STSTZI  XITKIPACB 
90  STAIDAIDXZID  DETflOPSZlT  IAIDIAIS  ..... 
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SUMMARY  STATISTICS  FROM  EVALUATIONS 
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CI6_SI_S££2 


PARE 

#13 

Auto. Reg. Gen 

•  1 

For. Reg. Spec 

798.2 

LABE 

#13 

Auto. Beg. Gen 

•  1 

For . Beg. Spec 

772.2 

PSL/PSA 

•  13 

Auto. Beg. Gen 

•  1 

For. Reg. Spec 

808.6 

samp 

#13 

Auto. Reg. Gen 

•  1 

For. Beg. Spec 

595.4 

2974.4 

C  AVS 

•  20 

Sof t.Std 

•  2 

QA . Procs&Guides 

723.6 

PAVS 

•  20 

Soft. Std 

•  2 

QA. ProcsSGuides 

ipJb.a 

FTN  77  AHA 

•  20 

Soft.Std 

•  2 

QA. ProcsGGuides 

720.0 

2400.4 

RABfilS-SES 

•  4 

Std.Sa.Hult.Env’s 

•  3 

Int. Sys. Acc 

900.0 

PDP  1  1/UNIX 

•  4 

Std.Sa.Hult.Env's 

•  3 

Int.Sys.Acc 

840.0 

SBL-SFTOOL80 

•  4 

Std. Sa.Hult. Env's 

•  3 

Int. Sys. Acc 

870.0 

V  AX~ IS/ 1 

•  4 

Std.Sa.Hult.Env's 

*  3 

Int.Sys.Acc 

955.0 

3565.0 

HARRIS 

•  4 

Std.Sa.Hult.Env's 

•  4 

Incr.No.Tera's 

892.4 

PDP  11/70 

•  4 

Std.Sa.Hult.Env's 

•  4 

Incr. No. Term' s 

933.8 

SEL 

•  4 

Std. Sa. Hult. Env's 

•  4 

Incr.No.Tera's 

851.0 

VAX 

•  4 

Std. Sa. Hult . Env ' s 

•  4 

Incr.No.Tera’s 

952.2 

3629.4 

RDP  1100 

#13 

Auto. Reg. Gen 

•  5 

Reguireaents  Tracking 

378.4 

378.4 

FASP 

•  1 

Int. Spt. Dev. Sys 

•  9 

Conf .Cntl 

672.0 

IS/1  PUB 

•  1 

Int.Spt.Dev.Sys 

»  9 

Conf .Cntl 

816.0 

SOFTOOL  II 

•  1 

Int. Spt. Dev. Sys 

•  9 

Conf . Cntl 

644.0 

SOLID 

•  1 

Int.Spt.Dev.Sys 

•  9 

Conf. Cntl 

560.0 

DNIX 

•  1 

Int.Spt.Dev.Sys 

•  9 

conf .Cntl 

756.0 

CCS 

•  5 

Conf.Cntl.Sys 

#  9 

Conf .Cntl 

860.0 

sees 

•  5 

Conf .Cntl . Sys 

#  9 

Conf .Cntl 

968.0 

SLIB 

•  5 

Conf.Cntl.Sys 

•  9 

Conf . Cntl 

741*. 0 

SHS 

•  5 

Conf .cntl. Sys 

•  9 

Conf .Cntl 

464.0 

SOFTOOL  II 

»  5 

Conf.Cntl.Sys 

•  9 

Conf. Cntl 

628.0 

SPHS 

•  5 

Conf.Cntl.Sys 

•  9 

Conf . Cntl 

824.0 

7936.0 

OPTIHA 

•  7 

Pro j.Hgt.Sys 

•  10 

Inp.Hilest.Id 

663.0 

CPAT 

•  9 

Prj. Pth. Ana. Hth 

•  10 

lap. Hilest. Id 

519.0 

CPH 

•  9 

Prj. Pth. Ana. Hth 

•  10 

Iap.Hilest.Id 

375.0 

PERT 

*  9 

Prj. Pth. Ana. Hth 

•  10 

lap. Hilest. Id 

489.0 

2046.0 

IS/1  INaail 

•  1 

Int. Spt . Dev . Sys 

*11 

Decr.Ppr'vk 

432.0 

ONIVAC-UNADS 

•  3 

Sg.Lg.Hult-Us.Env 

•  11 

Deer. Ppr' vk 

352.0 

IS/1  INaail 

•  6 

Autoaated  Off 

•  11 

Decr.Ppr'vk 

318.0 

CPAT 

t  7 

Pro j. Hgt .Sys 

•  11 

Decr.Ppr'vk 

410.0 

OPTIHA 

•  7 

Proj.Hgt.Sys 

•  11 

Decr.Ppr'vk 

442.0 

RDP  1100 

•  7 

Pro j.Hgt.Sys 

•  11 

Deer. Ppr ' vk 

322.0 

SCERT  II 

•  7 

Proj.Hgt.Sys 

•  11 

Decr.Ppr'vk 

388.0 

IS/1  INvord 

•  16 

Int. Txt. Proc 

#11 

Deer. Ppr • vk 

376.0 

0TS4K  PROC 

•  16 

Int.Txt.Proc 

•  11 

Decr.Ppr'vk 

370.0 

D'ly  Planlt 

•  17 

Auto. Data. Coll 

•  11 

Deer. Ppr' vk 

466.0 

3876.0 

OPTIHA 

•  7 

Proj.Hgt.Sys 

•  12 

laprove  Ranloading 

618.8 

PRICE 

•  7 

Proj.Hgt.Sys 

•  12 

Iaprove  Ranloading 

546.0 

SCERT  II 

•  7 

Proj.Hgt.Sys 

•  12 

laprove  Ranloading 

543.2 

SLIH 

•  7 

Pro j . Hgt. Sys 

•  12 

Iaprove  Ranloading 

532.0 
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COCOHO 

* 

8 

Cost. Est. S^s 

•  12 

Inprove  Hanloading 

378.0 

PRICE 

* 

8 

Cost . Est. Sys 

•  12 

Isprove  Hanloading 

532.0 

SLIH 

• 

8 

Cost. Bst.Sys 

•  12 

Inprove  Hanloading 

515.2 

CPH 

• 

9 

Pr j. Pth. Ana. nth 

•  12 

Isprove  Hanloading 

475.0 

PEBT 

1 

9 

Pr j. Pth. Ana. nth 

•  12 

Isprove  Hanloading 

456.4 

SCEBT  II 

• 

9 

Pr j. Pth. Ana. nth 

•  12 

Isprove  Banloading 

442.4 

ASET 

•  1 

2 

Auto.Trng. Pgs 

•  12 

Isprove  Hanloading 

366.8 

5405.8 

CPAT 

• 

7 

Proj.Hgt.Sys 

•  14 

Ispr .Schd.Ispc. Ana 

533.0 

OPTIHA 

• 

7 

Pro j.Hgt.Sys. 

•  14 

Ispr.Schd.Iapc.Ana 

574.6 

PRICE 

• 

7 

Proj.Hgt.Sys 

•  14 

Ispr . Schd. Ispc. Ana 

507.0 

SCEBT  II 

• 

7 

Proj.Hgt.Sys 

•  14 

Ispr. Schd.Ispc. Ana 

504.4 

sun 

• 

7 

Pro j. Hgt .Sys 

•  14 

Ispr. Schd. Ispc. Ana 

494.0 

cocono 

• 

8 

Cost. Est. Sys 

•  14 

Ispr . Schd. Ispc. Ana 

351.0 

PRICE 

• 

8 

Cost. Est. Sys 

•  14 

Ispr. Schd. Ispc. Ana 

494.0 

sun 

• 

8 

Cost. Bst.Sys 

•  14 

Ispr .Schd.Ispc.  Ana 

478.4 

CPAT 

t 

9 

Pr j. Pth. Ana. nth 

•  14 

Ispr. Schd. Ispc.  Ana 

449.8 

CPH 

t 

9 

Pr j.Pth.Ana.Hth 

•  14 

Ispr.Schd.Iapc.Ana 

325.0 

PEBT 

• 

9 

Pr j. Pth. Ana. Hth 

•  14 

Ispr.Schd.Iapc.Ana 

423.8 

5135.0 

fasp 

• 

1 

Int. Spt. Dev. Sys 

•  16 

Op. Old. Doc 

470.4 

SOFTOOL  80 

• 

1 

Int.Spt.Dev.Sys 

•  16 

Op. Old. Doc 

599.2 

1069.6 

DUAL 

• 

2 

High-Order  Lang 

•  18 

Fstr. Int. New. Espl 's 

411.4 

FORTRAN  77 

» 

2 

High-Order  Lang 

•  18 

Fstr. Int. New. Ea pi's 

572.0 

HIPERGRPH 

»10 

Sft.Eng.Prt.Trg 

•  18 

Fstr. Int . New. Eapl • s 

279.4 

ASET 

112 

Auto.Trng. Pgs 

•  18 

Fstr. Int. New. Eapl 's 

366.8 

IPF 

#15 

SPF 

•  18 

Fs tr. Int. New. Eapl 's 

272.8 

FEDSIH 

#23 

User. Asst. Func 

•  18 

Fstr. Int. New. Espl 's 

143.0 

2045.4 

CS4 

#11 

Rapid  Prototype 

•  21 

Siaulator  for  Design 

355.2 

PAHS 

#11 

Rapid  Prototype 

•  21 

Siaulator  for  Design 

310.4 

USEIT 

#11 

Rapid  Prototype 

•  21 

Siaulator  for  Design 

400.0 

1065.6 

CS4 

•  11 

Rapid  Prototyping 

•  22 

PDL 

799.2 

SHELL 

#11 

Rapid  Prototype 

•  22 

PDL 

835.2 

OSEIT 

#11 

Rapid  Prototype 

*22 

PDL 

900.0 

PDL 

•  1U 

Soft. Dsqn. Lang 

•  22 

PDL 

921.6 

SDDL 

•  14 

Soft. Dsgn. Lang 

•  22 

PDL 

1119.6 

4575.6 

IS/1  INword 

• 

1 

Int.Spt.Dev.Sys 

•  34 

Auto. Txt. Hgt. Sys 

767.6 

IS/1  INed 

• 

6 

Automated  Off 

•  34 

Auto. Txt. Hgt . Sys 

611.8 

IPF 

#15 

SPF 

•  34 

Auto. Txt. Hgt. Sys 

471.2 

IS/1  INword 

•  16 

In t . Txt . Proc 

•  34 

Auto. Txt. Hgt. Sys 

714.4 

0TS4K  PROC 

•  16 

Int.Txt.Proc 

•  34 

Auto. Txt. Hgt. Sys 

703.0 

3268.0 

KARRIS 

• 

4 

Std.Sa.Hult.Env's 

•  36 

Graphics  Aids 

561.0 

SEL 

• 

4 

Std.Ss.Hult.Env's 

•  36 

Graphics  Aids 

727.6 

TAX 

• 

4 

Std.Ss.Hult.Env's 

•  36 

Graphics  Aids 

727.6 

2016.2 

DAPPER 

• 

7 

Proj.Hgt.Sys 

•  40 

Hist. DB. Tech's 

233.2 

O'ly  Planlt 

#17 

Auto. Data. Coll 

•  40 

Hist. DB. Tech's 

605.8 

839.0 

ADA 

• 

2 

High-Order  Lang 

•  41 

Org . Tools/Tech's.  Int 

452.2 

FORTRAN  77 

• 

2 

High-Order  Lang 

*4  1 

Org. Tools/Tech' s.  Int 

884.0 
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SOFTOOL  II 

#  5 

Conf .Cntl.Sys 

•  41 

Or g. Tools/Tech's. Int 

533.8 

PARE 

#13 

Auto. Req. Gen 

•  41 

Org. Tools/Tech's.  Int 

1043.8 

LABE 

#13 

Auto. Req. Gen 

•  41 

Org . Too Is /Tech • s. Int 

1009.8 

PSL/PSA 

#13 

Auto. Req. Gen 

•  41 

Org. Tools/Tech ' s. Int 

1057.4 

RDP  1100 

#13 

Auto. Req. Gen 

•  41 

org . Tools/Tech ' s.  Int 

584.8 

SRIHP 

•  13 

Auto. Req . Gen 

•  41 

Org . Tools/Tech ' s. Int 

778.6 

POL 

•  14 

Sof t . Dsqn . Lang 

#41 

Org. Tools/Tech 's. Int 

870.4 

SDDL 

•  14 

Soft. Dsgn. Lang 

•  41 

Org. Tools/Tech's. Int 

1057.4 

8272.2 

CCS 

•  5 

Conf. Cntl.Sys 

•  42 

User. Asst. Func 

774.0 

sees 

•  5 

Conf .Cntl.Sys 

•  42 

User. Asst. Func 

871.2 

SLIB 

#  5 

Conf .Cntl. Sys 

•  42 

User. Asst. Func 

669.6 

sns 

#  5 

Conf .Cntl. Sys 

•  42 

User. Asst. Func 

417.6 

SOFTOOL  II 

#  5 

Conf .Cntl. Sys 

•  42 

User. Asst. Func 

565.2 

SPHS 

#  5 

Conf. Cntl. Sys 

•  42 

User . Asst. Func 

74  1.6 

FEDSIB 

•  2  3 

Oset . Asst. Func 

•  42 

User . Asst. Func 

234.0 

4273.2 

PEDSIfl 

•  23 

User . Asst. Func 

•  44 

Error  Rate  Standards 

169.0 

169.0 

D'  ly  Planlt 

#17 

Auto. Data. Coll 

•  46 

Red. Acct. Data. Rept.Anoa 

652.4 

652.4 

HYPERGRPH 

#10 

Sft.Eng.Prt.Trg 

•  47 

Coap. Trng. Pga 

482.6 

ASET 

#12 

Auto.Trng. Pga 

•  47 

Coap.Trng.Pga 

497.8 

SOFTOOL  80 

#12 

Auto.Trng. Pga 

•  47 

Coap. Trng. Pga 

425.6 

1406.0 

Planlt  BlBk 

#21 

Chargeback  Systea 

•  48 

Chargeback  Systea 

465.8 

465.8 

HARRIS 

•  4 

Std.Se. Hult.Env's 

*52 

Deer. Turn. Tiae 

659.6 

PDP  11/70 

•  4 

Std.Sa. Hult. Env's 

•  52 

Deer. Turn. Time 

690.2 

SEL 

•  4 

Std.Sa. Hult.Env's 

•  52 

Deer. Turn. Tiae 

629.0 

VAX 

•  4 

S td.Sa. Bui t. Env's 

•  52 

Deer. Turn. Tiae 

703.8 

703.8 

UIFOLA 

#14 

Soft. Dsgn. Lang 

•  54 

Natl. Lang. User /Sys. Int 

267.4 

267.4 

UNIVAC-UK'S 

#  3 

Sg.Lg.Hult-us.Env 

•  55 

Hod. Src. Data. Ent. Tech' s 

1021.2 

IS/1 

•  4 

Std.Sa. Hult. Env's 

•  55 

Hod. Sr c. Data. Ent. Tech 's 

910.8 

SES 

•  4 

Std.Sa. Hult.Env's 

•  55 

Hod . Src. Data . Ent. Tech 's 

837.2 

UNIX 

•  4 

Std.Sa. Hult.Env's 

•  55 

Hod.  src.  Data. Ent. Teel- 1 s 

800.4 

IPF 

#15 

SPF 

•  55 

Hod. Src. Da t a. Ent. Tech 's 

570.4 

4140.0 

BAPPER 

#  7 

Proj.Bgt.Sys 

•  56 

Hgt. Trkg. Pune's 

902.4 

OPTIHA 

#  7 

Pro j . Hgt. Sys 

•  56 

Hgt. Trkg. Func's 

707.2 

PRICE 

•  7 

Proj.Bgt.Sys 

*56 

Hgt. Trkg. Func's 

624.0 

RDP  1100 

#  7 

Pro j. Hgt .Sys 

•  56 

Hgt. Trkg. Func's 

515.2 

SCRRT  II 

•  7 

Proj.Bgt.Sys 

•  56 

Hgt. Trkg. Func's 

620.8 

SLIB 

•  7 

Pro j. Hgt .Sys 

•  56 

Hgt. Trkg. Func's 

608.0 

COCOHO 

•  e 

Cost. Est. Sys 

#56 

Hgt. Trkg. Func's 

432.0 

PRICE 

•  8 

Cost.Est. Sys 

•  56 

Hgt. Trkg. Func's 

608.0 

SLIB 

•  8 

Cost. Est. Sys 

#56 

Hgt. Trkg. Func's 

588.8 

CPAT 

•  9 

Pr j. Pth. Ana. nth 

•  56 

Hgt. Trkg. Func's 

553.6 

CPB 

•  9 

Pr j.Pth. Ana. Hth 

•  56 

Hgt. Trkg. Func' s 

400.0 

PERT 

#  9 

Pr j.Pth.Ana.Hth 

•  56 

Hgt. Trkg. Func's 

521.6 

SCE9T  II 

*  9 

Pr j.Pth. Ana. Bth 

•  56 

Hgt. Trkg. Func's 

505.6 

7587.2 

FASP 

•  1 

Int. Spt. Dev. Sys 

•  57 

Soft. Dev. Tools 

705.6 

* 
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IS/1  PUB 

#  1 

In t . Spt . Dev . Sys 

•  57 

Soft. Dev. Tools 

886.2 

SOFTOOL  80 

1  1 

Int.  Spt. Dev. Sys 

*57 

Sof t. Dev. Tools 

898.8 

SOLID 

*  1 

Int. Spt. Dev. sys 

#57 

Sof t. Dev. Tools 

588.0 

ADA 

•  2 

High-Order  Lang 

#57 

Soft. Dev. Tools 

558.6 

DUAL 

*  2 

High-Order  Lang 

*57 

Soft. Dev. Tools 

785.4 

FORTRAN  77 

#  2 

High-Order  Lang 

#57 

Soft. Dev. Tools 

1050.0 

CS4 

#11 

Sapid  Prototyping 

•  57 

Soft. Dev. Tools 

932.4 

PAHS 

#11 

Rapid  Prototyping 

#57 

Soft. Dev. Tools 

814.8 

OSEIT 

#11 

Rapid  Prototyping 

#57 

Soft. Dev. Tools 

1050.0 

PARE 

#13 

Auto. Req. Sen 

•  57 

Soft. Dev. Tools 

1289.4 

L  A  R  E 

#13 

Auto. Req. Sen 

#57 

Soft. Dev. Tools 

1247. 4 

PSL/PSA 

•  13 

Auto. Req. Sen 

•  57 

Soft. Dev. Tools 

1306.2 

RDP  1100 

#13 

Auto. Req. Sen 

•  57 

Soft. Dev. Tools 

722.4 

SRIHP 

#13 

Auto. Req. Sen 

•  57 

Sof t . Dev. Tools 

961.8 

ADF 

•  1U 

Soft. Dsgn. Lang 

•  57 

Soft. Dev. Tools 

684.6 

PDL 

#14 

Soft. Dsgn. Lang 

•  57 

Soft. Dev. Tools 

1075.2 

SDDL 

•  14 

Soft. Dsgn. Lang 

#57 

Soft. Dev. Tools 

1306.2 

IPF 

•  15 

SPF 

#57 

Soft. Dev. Tools 

520.8 

FA  VS 

#19 

Soft. Test. Sys 

•  57 

Soft. Dev. Tools 

1289.4 

FTN  77  ANA 

#19 

Soft. Test. Sys 

#57 

Soft. Dev. Tools 

898.8 

SCHS 

#19 

Soft. Test. Sys 

•  57 

Sof t . Dev. Tools 

1134.0 

SOFTOOL  80 

#19 

Soft. Test. Sys 

#57 

Sof t . Dev. Tools 

1121.4 

CAVS 

#20 

Soft. Std 

#57 

Soft. Dev. Tools 

844.2 

FAVS 

#20 

Sof t. Std 

#57 

Sof t. Dev. Tools 

1209.6 

FTN  77  ANA 

#20 

Soft. Std 

#57 

Sof t. Dev .Tools 

840.0 

SOFTOOL  80 

•  1 

Int. Spt. Dev. Sys 

•  58 

Prod. Pga. opt 

24721.0 

856.0 

CAVS 

#19 

Soft. Test. Sys 

#58 

Prod. Pga. Opt 

784.0 

FAVS 

#19 

Sof t. Test. Sys 

•  58 

Prod. Pga. Opt 

1228. 0 

FTN  77  ANA 

#19 

Sof t. Test. Sys 

*58 

Prod. Pga. Opt 

856.0 

SCHS 

#19 

Sof t. Test. Sys 

#58 

Prod . Pga. opt 

1080.0 

SOFTOOL  80 

#19 

Sof t. Test. Sys 

#58 

Prod. Pga. Opt 

1068.0 

5872.0 

APSE 

•  1 

Int. Spt. Dev. Sys 

*59 

Std . Phsd . Dev 

457.2 

ADA 

•  22 

Structured  Pqn 

#59 

Std. Phsd . Dev 

414.0 

871.2 

UNIVAC  11/62 

•  3 

Sg.Lg.Hult-0s.Env 

•  60 

Std . Dev . Hd ' vr 

843.6 

HARRIS 

•  4 

Std.Sa.Rult.Env's 

#60 

Std . Dev . Hd' vr 

737.2 

SEL 

•  4 

Std.Sa.Rult.Env's 

#60 

Std. Dev. Hd*  vr 

703.0 

VAX 

•  4 

Std.Sa.Rult.Env's 

•  60 

Std. Dev. Hd ' vr 

786.6 

3070.4 
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ADA 

•  2 

High-Order  Lang 

•  41 

Org. Tools/Tech ' s.  Int 

452.2 

ADA 

•  2 

High-Order  Lang 

•  57 

Sof t . Dev. Tools 

558.6 

ADA 

•  22 

Structured  Pga 

•  59 

Std.Phsd.Dev 

414.0 

1424.8 

ADF 

•  14 

Soft. Dsgn. Lang 

*57 

Sof t. Dev. Tools 

684.6 

684.6 

IPSE 

•  1 

Int. Spt. Dev. Sys 

*59 

Std.Phsd.Dev 

457.2 

457.2 

ASET 

•  12 

Auto. Trng . Pga 

•  12 

Iaprove  Hanloading 

366.8 

ASET 

•  12 

Auto. Trng. Pga 

•  18 

Pstr. Int. Hev.Eapl's 

366.8 

ASET 

*12 

Auto. Trng. Pga 

•  47 

Coap. Trng. Pga 

497.8 

1231.4 

CAVS 

•  19 

Sof t . Test .Sys 

•  58 

Prod. Pga. Out 

784.0 

CAVS 

•  20 

Soft.Std 

•  2 

QA. Procs&Guides 

723.6 

CAVS 

•  20 

Sof  t . Std 

•  57 

Soft. Dev. Tools 

844.2 

2351.8 

CCS 

•  5 

Conf .Cntl.Sys 

•  9 

Conf .Cntl 

860.0 

CCS 

•  5 

Conf . Cntl . Sys 

•  42 

Oser. Asst. Pune 

774.0 

1634.0 

cocono 

•  8 

Cost. Est. Sys 

•  12 

Iaprove  Hanloading 

378.0 

cocono 

•  8 

Cost. Est. Sys 

•  14 

Iapr . Schd. I ape.  Ana 

351.0 

cocono 

•  8 

Cost. Est. Sys 

•  56 

Hgt.Trkg. Pune's 

432.0 

1161.0 

CPAT 

•  7 

Proj.Hgt.Sys 

•  11 

Decr.Ppr'vk 

410.0 

CPAT 

•  7 

Proj.flgt.Sys 

•  14 

Iapr . Schd. Tape.  Ana 

533.0 

CPAT 

•  9 

Prj. Pth. Ana. nth 

•  10 

lap. Hilest. Id 

519.0 

CPAT 

•  9 

Pr  j . Pth . Ana . Bth 

•  14 

Iapr . Schd. Iapc.  Ana 

449.8 

CPAT 

•  9 

Pr  j  .  Ptii .  Ana .  Hth 

•  56 

Hgt. Trig. Pune's 

553.6 

2465.4 

CPH 

•  9 

Prj. Pth. Ana. Bth 

•  10 

lap. Hilest. Id 

375.0 

CPH 

•  9 

Prj. Pth. Ana. »th 

•  12 

Iaprove  Hanloading 

475.0 

CPH 

•  9 

Prj. Pth. Ana. Hth 

•  14 

Iapr. Schd. Iapc. Ana 

325.0 

cpn 

•  9 

Pr j . Pth. Ana . Hth 

*56 

Hgt.Trkg. Pune's 

400.0 

1575.0 

CS4 

*11 

Hapid  Prototype 

*21 

Siaulator  for  Design 

355.2 

CS4 

*11 

Sapid  Prototyping 

•  22 

PDL 

799.2 

CS4 

•  11 

Rapid  Prototyping 

•  57 

Soft. Dev. Tools 

932.4 

2086.8 

D'  ly 

Planlt 

•  17 

Auto. Data. Coll 

•  11 

Deer.  Ppr '  v(c 

466.0 

O'  ly 

Planlt 

*17 

Auto. Data. Coll 

140 

Hist. DB. Tech's 

605.8 

D'  ly 

Planlt 

*17 

Auto. Data. Coll 

*46 

Red. Acct.Data. Rept. Anoa 

652.4 

1724.2 

DUAL 

•  2 

High-Order  Lang 

•  18 

Pstr. Int. Nev.Ea pi's 

411.4 

DUAL 

•  2 

High-Order  Lang 

•  57 

Soft. Dev. Tools 

785.4 

1196.8 

FAHE 

•  13 

Auto. Req. Gen 

t  1 

For. Req. Spec 

798.2 

PAHB 

•  13 

Auto. Reg. Gen 

•  41 

Org. Tools/Tech's. Int 

1043.8 

PAHE 

•  13 

Auto. Reg. Gen 

•  57 

Sof t. Dev. Tools 

1289.4 

3131.4 

PASP 

•  1 

Int. Spt. Dev. sys 

•  9 

Conf .Cntl 

672.0 

PASP 

•  1 

Int. Spt. Dev. Sys 

•  16 

Dp. Old. Doc 

470.4 

PASP 

•  1 

Int. spt. Dev. sys 

*57 

Soft. Dev. Tools 

705.6 

1848.0 
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PAVS 

•  19 

soft. Test. Sys 

PAVS 

•  19 

Soft. Test.  Sys 

PAVS 

•  20 

Soft.Std 

PAVS 

•  20 

Soft.Std 

PEDSIH 

•  23 

Oser. Asst. Fuse 

PEDSIH 

•23 

User. Asst. Pune 

PEDSIH 

•  23 

Osec. Asst. Pune 

FORTRAN  77 

•  2 

High-Order  Lang 

FORTRAN  77 

•  2 

High-Order  Lang 

FORTRAN  77 

•  2 

High-Order  Lang 

PTN  77  ANA 

•  19 

Soft. Test. Sys 

PIN  77  ANA 

•  19 

Soft. Test. Sys 

PTN  77  ANA 

•  20 

Soft.Std 

PTN  77  ANA 

•  20 

Soft.Std 

HARRIS 

•  4 

Std.Sn.Hult.Env's 

HARRIS 

•  U 

Std.Sa.Hult.Env's 

HARRIS 

•  U 

Std.Sa.flult.Env's 

HARRIS 

•  4 

Std.Sa.Hult.Env's 

HARRIS-SES 

t  4 

Std.Sa.Hult.Env's 

HYPERGRPH 

•  10 

Sf t.Eng.Prt.Trg 

HYPERGRPH 

•  10 

Sft.Eng.Prt.Trg 

IPP 

*15 

SPP 

IPP 

•  15 

SPP 

IPF 

*15 

SPP 

IPP 

•  15 

SPP 

IS/1 

•  4 

Std.Sa.Hult.Env's 

IS/1  INed 

•  6 

Autoaated  Off 

IS/1  INnail 

*  1 

Int.Spt. Dev. Sys 

IS/1  INmail 

•  6 

Autoaated  Off 

IS/1  INwocd 

•  1 

Int. Spt. Dev. Sys 

IS/1  INword 

•  16 

Int . Txt . Proc 

IS/1  INwocd 

•  16 

Int. Txt. Proc 

IS/1  PHB 

•  1 

Int.Spt. Dev. Sys 

IS/1  PHB 

•  1 

Int. Spt. Dev. Sys 

LARE 

•  13 

Auto. Reg. Gen 

LARE 

•  13 

Auto. Reg. Gen 

LARE 

•  13 

Auto. Beq. Gen 

HAPPER 

•  7 

Pro j.Hgt.Sys 

RAPPER 

•  7 

Proj.Hgt.Sys 

OPTIHA 

•  7 

Pro j.Hgt.Sys 

OPTIHA 

*  7 

Proj.Hgt.Sys 

•  57 

Soft. Dev. Tools 

1289.4 

•  58 

Prod. Pga. Opt 

1228.0 

•  2 

QA. Procs&Guides 

1036.8 

•  57 

Soft. Dev. Tools 

1209.6 

4736. B 

•  18 

Pstr.Int.New.Eapl's 

143.0 

•  42 

Oser. Asst. Pune 

234.0 

•  44 

Error  Rate  Standards 

169.0 

546.0 

•  18 

Pstr. Int. New. Ea pi's 

572.0 

•  41 

Or g. Tools/Tech's. Int 

884.0 

*57 

Soft. Dev. Tools 

1050.0 

2506.0 

•  57 

Soft. Dev. Tools 

898.8 

*58 

Prod. Pga. Opt 

856.0 

»  2 

QA. ProcsBGuides 

720.0 

•  57 

Soft. Dev. Tools 

840.0 

3314.8 

•  4 

Incr. No. Tern's 

892.4 

•  36 

Graphics  Aids 

561.0 

*52 

Deer. Turn. Tiae 

659.6 

•  60 

Std. Dev. Hd ’ wr 

737.2 

•  3 

Int. Sys. Acc 

900.0 

3750.2 

*18 

Pstr.Int.New.Eapl's 

279.4 

*47 

Coap. Trng. Pga 

482.6 

762.0 

•  18 

Pstr.Int.New.Eapl's 

272.8 

*34 

Auto.Txt.Hgt.Sys 

471.2 

*55 

Hod. Src. Data.Ent.Tech's 

570.4 

*57 

Soft. Dev. Tools 

520.8 

1835.2 

•  55 

Hod. Src. Data.Ent.Tech's 

910.8 

910.8 

•  34 

Auto.Txt.Hgt.Sys 

611.8 

611.8 

•  11 

Deer. Ppr • wk 

432.0 

•  11 

Decr.Ppr'wk 

318.0 

750.0 

•  34 

Auto.Txt.Hgt.Sys 

767.6 

*11 

Deer. Ppr' wk 

376.0 

•  34 

Auto.Txt.Hgt.Sys 

714.4 

1858.0 

*  9 

Conf .Cntl 

816.0 

•  57 

Soft. Dev. Tools 

886.2 

1702.0 

•  1 

For. Reg. Spec 

772.2 

•  41 

Or g. Tools/Tech 's. Int 

1009.8 

•  57 

Soft. Dev. Tools 

1247.4 

3029.4 

•  40 

Hist. DB. Tech's 

233.2 

•  56 

Hgt.Trkg. Pune's 

902.4 

1135.6 

•  10 

lap. Hilest.Id 

663.0 

•  11 

Deer. Ppr • wk 

442.0 
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OPTIMA 

*  7 

Proj.Mgt.Sys 

•  12 

Iaprove  Banloading 

618.8 

OPTIMA 

•  7 

Proj.Mgt.Sys. 

•  14 

Iapr. Schd. Impc. Ana 

574.6 

OPTIMA 

•  7 

Pro j. Hgt. Sys 

•  56 

Hgt. Trkg. Pune's 

707.2 

3005.6 

Planlt  BIBk 

•  21 

Chargeback.  System 

•  48 

Chargeback  System 

465.8 

465.8 

PAHS 

•  11 

Rapid  Prototype 

•  21 

simulator  for  Design 

310.4 

PAHS 

•  11 

Rapid  Prototyping 

•  57 

Soft. Dev. Tools 

814.8 

1125.2 

PDL 

•  14 

Soft. Dsgn.Lang 

•  22 

PDL 

921.6 

POL 

•  14 

Soft.Dsgn. Lang 

•  41 

Or g. Tools/Tech' s. Int 

870.4 

PDL 

•  14 

Soft. Dsgn.Lang 

*57 

Soft. Dev. Tools 

1075.2 

2867.2 

PDP  11/ONIX 

•  4 

Std.Sm.Hult.Env's 

•  3 

Int. Sys. Acc 

840.0 

840.0 

PDP  11/70 

•  4 

Std.Sm.Hult.Env's 

•  4 

Incr. No. Term's 

933.8 

PDP  11/70 

•  4 

Std.Sa.Hult.Env's 

•  52 

Deer. Turn. Time 

690.2 

1624.0 

PERT 

•  q 

Pr j. Pth. Ana. Mth 

*10 

Imp.Kilest.ld 

489.0 

PERT 

•  9 

Pr j. Pth. Ana. Hth 

•  12 

Improve  Manloading 

456.4 

PERT 

•  9 

Pr j. Pth. Ana. Mth 

•  14 

Iapr.Schd. Impc.  Ana 

423.8 

PERT 

•  9 

Pr j. Pth. Ana. Mth 

•  56 

Hgt. Trkg. Pune's 

521.6 

1890.8 

PRICE 

•  7 

Proj.Mgt.Sys 

*12 

Improve  Manloading 

546.0 

PRICK 

•  7 

Proj.Mgt.Sys 

•  14 

Impr. Schd. Impc. Ana 

507.0 

PRICE 

«  7 

Proj.Mgt.Sys 

•  56 

Mgt. Trkg. Pune's 

624.0 

PRICE 

•  8 

Cost. Est. Sys 

•  12 

Improve  Manloading 

532.0 

PRICE 

•  8 

Cost. Est. Sys 

•  14 

Impr. Schd. Impc. Ana 

494.0 

PRICE 

•  8 

Cost. Est. Sys 

*56 

Mgt. Trkg. Pune's 

608.0 

3311.0 

PSL/PSA 

113 

Auto. Req. Gen 

•  1 

Por.Beq.Spec 

808.6 

PSL/PSA 

•  13 

Auto. Reg. Gen 

•  41 

Org. Tools/Tech's. Int 

1057.4 

PSL/PSA 

•  13 

Auto. Req. Gen 

•  57 

Soft. Dev. Tools 

1306.2 

3172.2 

RDP  1100 

•  7 

Pro j. ngt . Sys 

*11 

Deer. Ppr ' vk 

322.0 

RDP  1100 

•  7 

Proj.Mgt.Sys 

•  56 

Hgt. Trkg. Pune's 

515.2 

RDP  1100 

•  13 

Auto. Req. Gen 

•  5 

Requirements  Tracking 

378.4 

RDP  1100 

*13 

Auto. Req. Gen 

*41 

Org. Tools/Tech's. Int 

584.8 

RDP  1100 

•  13 

Auto. Req. Gen 

•  57 

Soft. Dev. Tools 

722.4 

2522.8 

sees 

•  5 

Conf.Cntl.Sys 

•  9 

Conf .Cntl 

968.0 

sees 

•  5 

Conf .Cntl. Sys 

•  42 

User. Asst. Pune 

871.2 

1839.2 

SCERT  II 

•  7 

Pro j. ngt .sys 

*11 

Decr.Ppr'vk 

388.0 

SCBRT  II 

•  7 

Proj.Mgt.Sys 

•  12 

Improve  Manloading 

543.2 

SCERT  II 

t  7 

Proj.Hgt.  Sys 

•  14 

Impr. Schd. Impc. Ana 

504.4 

SCERT  II 

•  7 

Proj.Mgt.Sys 

•  56 

Hgt. Trkg. Pune's 

620.8 

SCERT  II 

t  9 

Pr j. Pth. Ana. Mth 

*12 

Improve  Manloading 

442.4 

SCBRT  II 

•  9 

Pr j. Pth. Ana. Hth 

•  56 

Hgt. Trkg. Pune's 

505.6 

3004.4 

SCHS 

•  19 

Soft. Test. Sys 

•  57 

Soft. Dev. Tools 

1134.0 

SCRS 

•  19 

Soft. Test. Sys 

•  58 

Prod. Pgm. Opt 

1080.0 

2214.0 

SDDL 

*14 

Soft. Dsgn .Lang 

•  22 

PDL 

1119.6 

SDDL 

•  14 

Soft. Dsgn.Lang 

*4  1 

Org. Tools/Tech 's.  Int 

1057.4 
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SDDL 

»1U 

Soft. Dsgn. Lang 

•  57 

Soft. Dev. Tools 

1306.2 

3483.2 

SEL 

•  « 

Std.Sa.Hult. Env's 

•  4 

Incr.Vo.Tera's 

851.0 

SEL 

•  4 

Std. Sa. Hult. Env ' s 

*36 

Graphics  Aids 

727.6 

SEL 

•  4 

Std.Sa.Hult.EnT's 

•  52 

Deer. Turn. Tiae 

629.0 

SE- 

•  4 

Std. Sa. Halt. Env's 

•  60 

Std. Dev. Hd*  vr 

703.0 

2910.6 

SBL-SPTOOL80 

•  4 

Std . Sa. Balt. Env 's 

•  3 

Int. Sys. Acc 

870.0 

870.0 

SES 

•  4 

Std.Sa.Hult.EnT's 

•  55 

Hod . Src. Data. Ent. Tech' s 

837.2 

837.2 

SHELL 

•  11 

Rapid  Prototype 

•  22 

PDL 

835.2 

835.2 

SLIB 

•  5 

Conf .Cntl.Sys 

•  9 

Conf .Cntl 

744.0 

SLIB 

#  5 

Conf .Cntl. Sys 

•  42 

Dser. Asst. Func 

669.6 

1413.6 

slih 

•  7 

Proj.Hgt.Sys 

•  12 

Iaprove  Hanloading 

532.0 

SLIH 

•  7 

Proj . Hgt. Sys 

•  14 

Iapr. Sc hd. I ape. Ana 

494.0 

slih 

•  7 

Pro  j . Hqt. Sys 

•  56 

Hgt.Trkg.Func's 

608.0 

sun 

t  8 

Cost. Est. Sys 

•  12 

Iaprove  Banloading 

515.2 

slih 

•  8 

Cost. Est. Sys 

•  14 

Iapr .Schd.Iapc. Ana 

478.4 

slih 

t  8 

Cost. Est. Sys 

*56 

Hgt.Trkg.Func's 

588.8 

3216.4 

sns 

•  5 

Conf .Cntl.Sys 

*  9 

Conf .Cntl 

464.0 

sns 

•  5 

Conf .Cntl.Sys 

•  42 

Oser. Asst. Func 

417.6 

881.6 

SOFTOOL 

II 

•  1 

Int. Spt. Dev. Sys 

•  9 

Conf -Cntl 

644.0 

SOFTOOL 

II 

•  5 

Conf .Cntl. Sys 

•  9 

Conf .Cntl 

628.0 

SOFTOOL 

II 

•  5 

Conf .Cntl.Sys 

•  41 

Org. Tools/Tech 's.  Int 

533.8 

SOFTOOL 

II 

•  5 

Conf .Cntl.Sys 

•  42 

Oser. Asst. Func 

565.2 

2371.0 

SOFTOOL 

80 

f  1 

Int. Spt. Dev. Sys 

*16 

Op. Old. Doc 

599.2 

SOFTOOL 

80 

•  1 

Int. Spt. Dev. Sys 

•  57 

Soft. Dev. Tools 

898.8 

SOFTOOL 

80 

•  1 

Int. Spt. Dev. Sys 

•  58 

Prod. Pga. Opt 

856.0 

SOFTOOL 

80 

•  12 

Auto.Trng. Pga 

*47 

Coap.Trng. Pga 

425.6 

SOFTOOL 

80 

*19 

Soft. Test. sys 

•  57 

Soft. Dev. Tools 

1121.4 

SOFTOOL 

80 

•  19 

Soft. Test. Sys 

*58 

Prod. Pga. Opt 

1068.0 

4969.0 

SOLID 

•  1 

Int. Spt. Dev. Sys 

•  9 

Conf .Cntl 

560.0 

SOLID 

•  1 

Int. Spt. Dev. Sys 

•  57 

Soft. Dev. Tools 

588.0 

1148.0 

spns 

»  5 

Conf .Cntl. Sys 

*  9 

Conf .Cntl 

824.0 

sphs 

*  5 

Conf .Cntl.Sys 

•  42 

Oser. Asst. Func 

741.6 

1565.6 

SRI  HP 

*13 

Aato. Beq.Gen 

•  1 

For. Reg. Spec 

595.4 

SRIHP 

•  13 

Auto. Reg. Gen 

•  41 

Org. Tool s/Tech's. Int 

778.6 

SRIHP 

•  13 

Auto. Req. Gen 

•  57 

Soft. Dev. Tools 

961.8 

2335.8 

OIFOLA 

•  14 

Soft. Dsgn . Lang 

•  54 

Natl. Lang. Oser /Sys. Int 

267.4 

267.4 

ONI  VAC 

11/62 

•  3 

Sg.Lg. Holt-Os. Env 

•  60 

Std . Dev. Hd' vr 

843.6 

843.6 

ONIVAC- 

ONADS 

•  3 

Sg.Lg.Hult-Os.Env 

•  11 

Deer .Ppr' v* 

352.0 

352.0 

ONIVAC- 

UK'S 

•  3 

Sg.Lg.Hult-0s.Env 

•  55 

Hod . Src. Data . Ent . Tech ' s 

1021.2 
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UNIX 

•  1 

Int.Spt.Dev.SfS 

•  4 

Conf .Cntl 

1021.2 

756.0 

UNIX 

t  4 

Std.Sa.Hult.Env's 

#55 

Hod. Src.Dat a. Rnt. Tech's 

800.4 

OSEIT 

•  11 

Rapid  Prototype 

#21 

Siaulator  for  Design 

1556.4 

400.0 

USBIT 

#11 

Rapid  Prototype 

#22 

PDL 

900.0 

OSEIT 

#11 

Rapid  Prototyping 

#57 

Soft. Dev. Tools 

1050.0 

UTS4K 

PROC 

•  16 

Int.Txt.Proc 

•  11 

Decr.Ppr* vk 

2350.0 

370.0 

0TS4K 

PROC 

#16 

Int.Txt.Proc 

•  34 

Aut0.Txt.flgt.S7s 

703.0 

VAX 

•  4 

Std.Sa.Hult.Env's 

•  4 

Incr. No.Tera's 

1073.0 

952.2 

VAX 

#  4 

Std.Sa.Hult.Env's 

•  36 

Graphics  Aids 

727.6 

VAX 

•  4 

Std.Sa.Hult.Env's 

•  52 

Deer. Turn. Tiae 

703.8 

VAX 

•  4 

Std.Sa.Hult.Env's 

•  60 

Std.Dev. Hd’ vr 

786.6 

3170.2 

£IE_1I_§£3I£ 

PSL/PSA 

#13 

Auto. Req. Sen 

#57 

Soft. Dev. Tools 

1306.2 

SDDL 

#14 

Soft. Dsqn. Lang 

#57 

Sof t. Dev. Tools 

1306.2 

FAHE 

#13 

Auto. Req. Sen 

#57 

Soft. Dev. Tools 

1289.4 

FAVS 

#19 

Soft. Test. Sys 

#57 

Soft. Dev. Tools 

1289.4 

LA  RE 

•  13 

Auto. Req. Sen 

#57 

Soft. Dev. Tools 

1247.4 

FAVS 

•  19 

Soft. Test. Sys 

#58 

Prod. Pgs. Opt 

1228.0 

FAVS 

#20 

Soft.Std 

#57 

Soft. Dev. Tools 

1209.6 

sens 

#19 

Soft. Test. Sys 

♦  57 

Soft. Dev. Tools 

1134.0 

SOFTOOL  60 

#19 

Soft. Test.  Sys 

#57 

Soft. Dev. Tools 

1121.4 

SOOL 

•  1U 

Soft. Dsqn. Lanq 

•  22 

PDL 

1119.6 

sens 

#19 

Soft. Test.  Sys 

#58 

Prod. Pgm.Opt 

1080.0 

PDL 

#  1  R 

Soft. Dsqn. Lang 
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APPENDIX  I 

LORAN  NAVIGATIONAL  LATTICE  PROBLEM 
FOR  USE. IT  EVALUATION 


original:  04  August  1982 
revised:  18  August  1982 
(Indicated  by  marginal  vertical  lines) 


DEMONSTRATION  USE. IT  PROBLEM  FOR 
DEFENSE  MAPPING  AGENCY  STUDY  (Contract  no.  F30602-81-C-0039) 
LORAN  NAVIGATIONAL  LATTICE  PROBLEM 


1.0  INTRODUCTION 

The  Navigational  Lattice  Problem  Is  a  typical  DMA  mapping 
problem  which  will  be  used  to  demonstrate  the  USE. IT  tool 
capabilities.  The  solution  to  the  problem  is  a  set  or 
lattice  of  hyperbolas  each  indicating  a  different  constant 
distance  from  a  master-slave  pair  of  radio  transmitting 
stations.  The  LORAN  operator  tunes  his  receiver  to  a 
master-slave  pair  of  stations  and  reads  a  time  delay  in 
microseconds.  This  time  delay  represents  the  time 
difference  in  receiving  the  radio  signal  from  the  master 
station  and  from  the  slave  station.  The  constant  distance 
is  directly  proportional  to  the  time  delay;  the 
proportionality  constant  being  the  radio  propagation 
velocity.  The  formula  used  to  determine  the  hyperbolas  is 

Vs  Time  B  ♦  Time  C  +  Time  D  -  Time  A,  where  V  is  a 
constant . 


As  an  example,  Figure  1  shows  that  the  ship  or  aircraft  is 
determined  to  be  on  the  1980  |isec  hyperbola  because 

Vs  (400  +1500  +260  -  180)psec  *  1980  psec 
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2.0  SCOPE  OF  PROBLEM  FOR  USE. IT  DEMONSTRATION 

The  purpose  of  the  LORAN  problem  is  to  demonstrate  the 
capabilities  of  the  USE. IT  tool.  Hence,  we  shall  assume  a 
simplified  mathematical  model  for  the  LORAN  line 
calculation,  and  emphasize  the  cartographic  aspects  of  the 
problem.  The  following  assumptions  simplify  the 
mathematical  model: 

1.  Assume  a  flat  earth. 

2.  Disregard  time  delay  complexltltes  caused  by  radio 
propagation  over  water  or  ground  and  reflections  from 
the  E  and  F  layers  of  the  ionosphere. 

3.  Consider  only  one  master-slave  pair  of  stations. 

We  will  employ  the  USE. IT  tool  to  produce  a  LORAN  chart  as 
shown  in  Figure  2. 


The  major  characteristics  of  this  chart  will  be: 

1.  Two  LORAN  stations  will  be  specified. 

2.  Smooth,  complete  hyperbolic  curves  will  be  drawn 
within  the  interior  hexagon. 

3.  The  points  of  intersection  of  the  LORAN  lines  with 
the  exterior  hexagons  will  be  shown. 

4.  The  hexagons  will  be  uniformly  spaced  and  concentric 
out  to  the  chart  boundary. 

5.  A  windowing  capability  will  be  provided  to  simulate 
the  placement  of  the  LORAN  data  on  a  paper  chart. 

6.  The  baseline  between  the  master  station  and  the  slave 
station  will  be  drawn  and  extended  to  the  chart  boundaries 
if  either  station  or  both  stations  are  within  the  chart 
boundary. 

7.  The  chart  boundary  will  be  a  rectangle  and  it  will  be 
drawn . 
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The  most  significant  input  data  are  planned  to  be: 

1.  Position  Definition  Parameters 

a.  Master  Station  Location  input  as  (latitude,  longitude) 

b.  Slave  Station  Location  input  as  (latitude,  longitude) 

2.  Cartographic  Option  Parameters 

a.  Chare  Window  Boundary  Points  input  as  (latitude,  longitude) 

b.  Hyperbolic  Curve  Spacing  in  chart  dimensions 

c.  Hexagon  Spaci^ 

d.  Overall  Scaling 

We  plan  to  demonstrate  the  following  two  scenarios: 

1.  On  the  HOS  VAX  system,  employ  USE. IT  for  requirements 
definition  and  Fortran  source  code  generation,  and 
employ  the  VAX  support  software  to  execute  the  code 
on  the  VAX  with  output  to  a  VT100  graphics  terminal. 

2.  On  the  UNIVAC  system,  demonstrate  the  compatibility 
of  U3E.IT  produced  Fortran  source  code  with  UNIVAC 
support  software  by  creating  a  UNIVAC  produced  magnetic 
tape  of  the  LORAN  data  that  is  subsequently  plotted 
off-line. 

The  Fortran  source  produced  by  the  RAT  function  of  USE. IT 
will  be  transported  to  a  UNIVAC  system  where  it  will  be 
compiled  and  executed.  In  a  like  manner  the  Fortran  source 
can  be  compiled  and  executed  on  the  VAX  system.  The  outputs 
differ  in  that  the  VAX  output  will  be  a  VT100  terminal  and 
the  UNIVAC  output  will  be  a  magnetic  tape,  for  subsequent 
off-line  plotting.  The  scenarios  are  shown  in  Figure  3- 

3-0  HIGH-LEVEL  TREE  STRUCTURE  FOR  THE  LORAN  PROBLEM 

Figure  4  shows  the  top-down,  high-level  tree  structure  for 
the  LORAN  problem.  This  figure  shows  the  requirements  for 
the  problem--not  its  implementation. 
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4.0  LIST  OF  CANDIDATE  HIGH-LEVEL  PRIMITIVES 

Within  the  context  of  the  LORAN  problem,  we  consider  a  high- 
level  primitive  to  have  the  characteristics  that  (1)  It 
perforas  a  generalized  function  In  the  requirements 
definition  of  a  problem,  and  (2)  Its  function  may  be  useful 
to  a  larger  class  of  DMA  probleas.  These  high-level 
primitives  could  fora  the  basis  of  a  library  of  functions 
that  would  be  tailored  tc  the  DMA  environment  and  make 
requirements  definition  eftsler  for  the  user.  The  following 
is  a  candidate  list  of  high-level  primitives  that  would  be 
developed  for  the  LORAN  problem: 

1.  DRAW  -  draws  a  straight  line  segment  between  two 
specified  points 

2.  TRANSLATE  -  translates  a  line  by  a  specified  displacement 
vector 

3.  ROTATE  -  rotates  a  line  in  two  dimensions  through  a 
specified  angle 

4.  SCALE  -  changes  the  dimensions  of  a  line  by  a  constant 
amount 

5.  WINDOW  -  selects  a  specified  two  dimensional  region  of  a 
larger  picture  for  display 

6.  PROJECTION  -  transforms  (latitude,  longitude)  to 
(abscissa,  ordinate) 

We  anticipate  that  these  high-level  primitives  would  be 
implemented  by  more  fundamental  primitives  that  would  also 
be  available  in  the  library. 

5.0  OUTPUT  FORMAT 

We  will  format  the  LORAN  data  for  two  output  devices:  (1)  a 
VT100  graphics  terminal  and  (2)  a  magnetic  tape  that  can  be 
read  by  the  UNIVAC  system.  (See  Figure  3  for  the  two 
demonstration  scenarios.) 

The  output  format  to  the  VT100  graphics  terminal  will  be 
compatible  with  the  VT100  graphics  terminal  and  the  HOS  VAX 
system.  Since  there  is  no  interface  with  the  UNIVAC  system 
for  this  scenario,  we  anticipate  no  problems. 

The  UNIVAC  scenario  requires  the  specification  of  two  output 
formats:  (1)  the  magnetic  tape  format  and  (2)  the  LORAN  data 
format.  The  magnetic  tape  format  for  the  Fortran  source 
code  will  be: 

9  tracks 

1600  bits  per  inch  density 
unlabelled 
unblocked 

80  column  card  images 
ASCII  format 
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1  Reference  Tape  A 
in  Figure  3 


« 


The  LORAN  data  will  be  temporarily  stored  In  an  array  A(2,201) 
as  follows: 


A ( 1 , 1 )  =  the  #  of  data  points 
A(2,1)  s  the  feature  index 

and  for  2  £  I  •£  201 


A  ( 1 , 1 ) 
A  ( 2 , 1 ) 


as  demonstrated  below. 


1 

2 

i 

200 

201 

*  •(  mat* 

*1 

*2 

.  .  . 

>200 

featere  lad* 

»2 

*  #  * 

TJOO 

Reference  Tape  B 
in  Figure  3 


Once  the  array  is  full  it  will  be  either  displayed  on  the 
screen  or  sent  to  a  tape  unit  using  an  unformatted  Fortran 
WRITE  statement  (see  tape  B  of  Figure  3)  depending  on 
whether  the  program  is  to  be  run  on  the  VAX  or  Unlvac. 


6.0  CANDIDATE  FOR  LORAN  PROBLEM  ENHANCEMENT 


To  demonstrate  the  capabilities  of  the  USE. IT  tool  for 
software  maintenance,  we  will  enhance  the  capabilities  of 
the  initial  LORAN  problem  solution  by  adding  Intermediate 
tick  marks  on  the  exterior  hexagons  of  the  LORAN  chart.  We 
will  incorporate  this  enhancement  at  the  requirements  level , 
and  we  will  add  another  high-level  primitive  TICK.  TICK  will 
place  tick  marks  on  a  specified  line  segment  at  a  specified 
interval.  The  intermediate  tick  marks  will  be  a  different 
length  from  the  hyperbola  intersection  tick  marks. 
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APPENDIX  J 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEETS 


CONCEPT  IHPLERENTATION  EVALUATION  SHEET 
o  I HPLEBE STATION:  IS/1  PBB 

o  FOB  CONCEPT:  II  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *9  Configuration  Control 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Docuaentation 

7 

2 

14 

Maturity 

9 

3 

27 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

3 

3 

9 

Environment  Compatibility 

5 

3 

15 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

software 

Conceptual  Simplicity 

7 

2 

14 

Use 

6 

2 

12 

Training 

System  Resources 

5 

3 

15 

Allocations  Required 

TOTAL 

NEED  WEIGHT 

CIS  SCORE 

6 

3 

18 

204.0 
x  4.0 
816.0 
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CONCEPT  INPLE DENTITION  EVALUATION  SHEET 


O  IHPLE DENTATION:  ONIZ 

o  FOR  CONCEPT:  *1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *9  Configuration  Control 


EVALUATION  CRITERIA 


EVALUATION  Z  HEIGHT  =  SCORE 


Interactive  Capability  10 

Support  Docunentation  4 

Maturity  9 

Vendor  Support  5 

Availability  8 

Hardware  Compatibility  10 

Environment  Compatibility  3 

Government  Access  3 

Flexibility  of  Use 

Hardware  5 

Software  5 

Conceptual  Simplicity 

Use  3 

Training  3 

System  Resources 

Allocations  Required  6 


3 

2 

3 

1 

3 

3 

3 

1 


30 

8 

27 

5 

24 

30 

9 

3 


2  10 
2  10 


2  6 

3  9 


3  18 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


189.0 
X  4.0 
756.0 
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CONCEPT  IMPLEMENTATION  ETALDATION  SHEET 
o  IMPLEMENTATION:  PASP 

o  FOH  CONCEPT:  #1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  #9  Configuration  Control 


EVALUATION  CRITEHIA 


EVALUATION  I  WEIGHT 


=  SCORE 


Interactive  Capability  5 

Support  Documentation  5 

Maturity  10 

Vendor  Support  8 

Availability  1 0 

Hardware  Compatibility  2 

Environment  Compatibility  2 

Government  Access  10 

Flexibility  of  Use 

Hardware  3 

Software  3 

Conceptual  Simplicity 

Use  7 

Training  5 

System  Resources 

Allocations  Required  4 


3 

2 

3 

1 

3 

3 

3 

1 


15 

10 

30 

8 

30 

6 

6 

10 


2  6 
2  6 


2  14 

3  15 


3  12 


TOTAL 

NEED  WEIGHT 
CIE  SCORE 


16  8.0 
X  4.0 
672.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  SOPTOOl  H 

o  FOB  CONCEPT:  »1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *9  Configuration  control 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 
Support  Documentation 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environment  Compatibility 
Government  Access 
Flexibility  of  use 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

System  Resources 

Allocations  Required 


TOTAL 

NEED  HEIGHT 
CIR  SCORE 


161.0 
I  4.0 
644.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  SOLID 

o  FOB  CONCEPT:  *1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *9  Configuration  Control 


EVALUATION  CRITERIA  .  EVALUATION  X  WEIGHT  =  SCORE 


Interactive  Capability 
Support  Documentation 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environment  Compatibility 
Government  Access 
Flexibility  of  use 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

System  Resources 

Allocations  Required 


9 

0 

8 

1 

1 

7 

5 

1 


3 

2 

3 

1 

3 

3 

3 

1 


27 

0 

24 

1 

3 

21 

15 

1 


8  2  16 
8  2  16 


5  2  10 

1  3  3 


1  3  3 


TOTAL 

NEED  WEIGHT 
Cl E  SCORE 


140.0 
X  4.0 
560.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  IS/1  INmail 

o  FOB  CONCEPT:  *1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  til  Decreased  Paperwork 


.  EVALUATION  CRITERIA 

.  EVALUATION  X 

HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

7 

2 

14 

Maturity 

9 

3 

27 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

4 

3 

12 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

Software 

Conceptual  Simplicity 

1 

2 

2 

Use 

9 

2 

18 

Training 

System  Resources 

9 

3 

27 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

7 

3 

21 

216.0 
x  2.0 
432.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IMPLEMENTATION:  SOFTOOL  BO 

o  FOB  CONCEPT:  #1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *16  Update  of  Old  Documentation 


EVALOATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 
Support  Documentation 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environaent  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

System  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


i 
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10 

3 

30 

10 

2 

20 

10 

3 

30 

8 

1 

B 

10 

3 

30 

5 

3 

15 

4 

3 

12 

5 

1 

5 

7 

2 

14 

7 

2 

14 

6 

2 

12 

8 

3 

24 

0  3  0 

214.0 
x  2.8 
599.2 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  FASP 

o  FOR  CONCEPT:  #1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  (16  Update  of  Old  Documentation 


EVALUATION  CRITBRIA 


EVALUATION  X  HEIGHT  -  SCORE 


Interactive  Capability 

5 

3 

15 

Support  Documentation 

5 

2 

10 

Maturity 

10 

3 

30 

Vendor  Support 

8 

1 

8 

Availability 

10 

3 

30 

Hardware  compatibility 

2 

3 

6 

Environment  Compatibility 

2 

3 

6 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

3 

2 

6 

Software 

Conceptual  Simplicity 

3 

2 

6 

Use 

7 

2 

14 

Training 

System  Resources 

5 

3 

15 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

4 

3 

12 

168.0 
x  2.8 
470.4 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  IS/1  INword 

o  FOB  CONCEPT:  *1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *34  Automated  Text  Management  Systea 


EVALUATION  CRITERIA 

.  EVALUATION  X  HEIGHT 

*  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

7 

2 

14 

Maturity 

9 

3 

27 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

3 

3 

9 

Environment  Compatibility 

4 

3 

12 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

Software 

Conceptual  Simplicity 

1 

2 

2 

Use 

8 

2 

16 

Training 

System  Resources 

7 

3 

21 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

7 

3 

21 

202.0 
x  3.8 
767.6 

CONCEPT  IflPLEHENTATION  EVALOATION  SHEET 
o  IflPLEBENTATION :  SOFTOOL  80 

o  FOB  CONCEPT:  *1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  »57  Software  Development  Tools 


EVALOATION  CRITERIA 


Interactive  Capability 
Support  Documentation 
flaturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environment  Compatibility 
Government  Access 
Flexibility  of  Ose 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

System  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 
CIS  SCORE 


EVALOATION 

X  HEIGHT 

=  SCORE 

10 

3 

30 

10 

2 

20 

10 

3 

30 

8 

1 

8 

10 

3 

30 

5 

3 

15 

4 

3 

12 

5 

1 

5 

7 

2 

14 

7 

2 

14 

6 

2 

12 

8 

3 

24 

0 

3 

0 

214.0 
X  4.2 

898.8 
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CONCEPT  IN PLEA ENT ATI ON  EVALUATION  SHEET 


O  IMPLEMENTATION:  EASP 

o  FOB  CONCEPT:  tl  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *57  Software  Development  Tools 

EVALUATION  CRITEHIA  .  EVALUATION  X  HEIGHT  =  SCOBE 


Interactive  Capability  5 

Support  Documentation  5 

Maturity  10 

Vendor  Support  8 

Availability  10 

Hardware  Coapatibility  2 

Environeent  Coapatibility  2 

Government  Access  10 

Flexibility  of  Use 

Hardware  3 

Software  3 

Conceptual  Simplicity 

Use  7 

Training  5 

System  Resources 

Allocations  Required  4 


3 

2 

3 

1 

3 

3 

3 

1 


15 

10 

30 

B 

30 

6 

6 

10 


2  6 
2  6 


2  m 

3  15 


3  12 


TOTAL 

NEED  HEIGHT 
C IE  SCORE 


168.0 
X  4.2 
705.6 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IBPLEHENTATION:  SOLID 

o  FOB  CONCEPT:  A1  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  *57  Software  Development  Tools 


EVALDATION  CRITERIA 


EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  9 

Support  Docuaentation  0 

Maturity  8 

Vendor  Support  1 

Availability  1 

Hardware  Coapatibility  7 

Environaent  Coapatibility  5 

Govetnaen*  Access  1 

Flexibility  of  Use 

Hardware  8 

Software  8 

Conceptual  Simplicity 

Use  5 

Training  1 

System  Resources 

Allocations  Required  1 


3 

2 

3 

1 

3 

3 

3 

1 


27 

0 

24 

1 

3 

21 

15 

1 


2  16 

2  16 

2  10 

3  3 


3  3 


TOTAL 

NEED  HEIGHT 
CIR  SCORE 


140.0 
X  4.2 

588.0 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


o  INPLEHENTATION:  IS/1  PUB 

o  FOB  CONCEPT:  II  Integrated  Support  Dev.  Sys . 
o  SATISFIES  NEED:  *57  Software  Development  Tools 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

»  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

6 

2 

12 

Naturity 

9 

3 

27 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

3 

3 

9 

Environment  Compatibility 

5 

3 

15 

Government  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

3 

2 

6 

Software 

7 

2 

14 

Conceptual  Simplicity 

Use 

6 

2 

12 

Training 

B 

3 

24 

System  Resources 

Allocations  Required 

6 

3 

18 

TOTAL 

211.0 

NEED  HEIGHT 

x4.2 

CIE  SCORE 

886.2 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  ADA  PROGRAMMING  SUPPORT  ENVIRONMENT 
o  FOR  CONCEPT:  II  Integrated  Support  Dev.  Sys. 
o  SATISFIES  NEED:  159  Standardized  Phased  Development 


EVALUATION  CRITERIA 


EVALUATION  X  HEIGHT  «  SCORE 


Interactive  Capability  10 

Support  Documentation  10 

Maturity  1 

Vendor  Support  9 

Availability  2 

Hardware  Compatibility  3 

Environment  Compatibility  3 

Government  Access  10 

Flexibility  of  Use 

Hardware  T 

Software  1 

Conceptual  Simplicity 

Use  6 

Training  2 

System  Resources 

Allocations  Required  3 


3 

2 

3 

1 

3 

3 

3 

1 


30 

20 

3 

9 

6 

9 

9 

10 


2  2 
2  2 


2  12 

3  6 


3  9 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


127.0 
x  3.6 
457.2 


PRECEDING  PAGE  BLANK-NOT  FILMED 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


0  IHPLEHENTATION:  FORTRAN  77 

o  FOR  CONCEPT:  »2  High-Order 

.  o  SATISFIES  NEED:  IIS  Faster 

Language 

Intergation  of  New 

Eapl's 

EVALUATION  CRITERIA 

EVALUATION 

X  HEIGHT 

=  SCORE 

Support  Docuaentation 

io 

2 

20 

Diagnostics 

Docuaentation 

8 

2 

16 

Interactive  Support 

6 

2 

12 

Haturity 

5 

3 

15 

Availability 

10 

3 

30 

Hardware  Coapatibility 

9 

3 

27 

Environaent  Coapatibility 

10 

3 

30 

Governaent  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

8 

2 

16 

'  Software 

9 

2 

18 

Conceptual  Siiplicity 

Use 

9 

2 

18 

Training 

9 

3 

27 

Systea  Resources 

Allocations  Required 

7 

3 

21 

TOTAL 

260.0 

NEED  HEIGHT 

x  2.2 

CIS  SCORE 

572.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  DUAL 
o  FOB  CONCEPT:  #2  High-Order  Language 

o  SATISFIES  NEED:  *18  Faster  Intergation  of  Nee  Eapl's 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Support  Docusentation 
Diagnostics 

0 

2 

0 

Docueentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Environment  Compatibility 

9 

3 

27 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

10 

2 

20 

Software 

Conceptual  Simplicity 

10 

2 

20 

Use 

5 

2 

10 

Training 

System  Resources 

0 

3 

0 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

5 

3 

15 

187.0 
x  2.2 
411.4' 
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UNCLASSIFIED 


INTERACTIVE  COMPUTER  PROGRAM  DEVELOPMENT  SYSTEM  STUDY 
VOLUME  1(U)  GENERAL  DYNAMICS  FORT  WORTH  TX  FORT  WORTH 
DIV  H  C  CONN  ET  Al .  JAN  83  DMA-2-014-V0L- 1 
RADC-TR-83-3-VOL- 1  F30602-81 -C-0039  F/G  9/2 


NL 


F/G  9/2 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


O  IBPLEHENTiTION:  FORTH AN  77 
o  FOB  CONCEPT:  *2  High-Order  Language 

o  SATISFIES  NEED:  *41  Organ.  Tools/Techniques  Interface 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Support  Docuaentation 

io 

2 

20 

Diagnostics 

Docuaentation 

8 

2 

16 

Interactive  Support 

6 

2 

12 

Haturity 

5 

3 

15 

Availability 

10 

3 

30 

Hardware  Coapatibility 

9 

3 

27 

Environaent  Coapatibility 

10 

3 

30 

Governaent  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

8 

2 

16 

Software 

9 

2 

18 

Conceptual  Siaplicity 

Use 

9 

2 

18 

Training 

9 

3 

27 

Systea  Resources 

Allocations  Required 

7 

3 

21 

TOTAL 

260.0 

NEED  HEIGHT 

x  3.4 

CIE  SCORE 

884.0 

290 


CONCEPT  I H PLEA ENT AT I ON  EVALUATION  SHEET 


O  IHPLEHENTATXON:  ADA 
o  FOB  CONCEPT:  #2  High-Order  Language 

o  SATISFIES  NEED:  #41  Organ.  Tools/Technigues  Interface 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Support  Documentation 
Diagnostics 

10 

2 

20 

Documentation 

10 

2 

20 

Interactive  Support 

8 

2 

16 

Haturity 

1 

3 

3 

Availability 

2 

3 

6 

Hardware  Compatibility 

3 

3 

9 

Environment  Compatibility 

3 

3 

9 

Government  Access 
Flexibility  of  Use 

10 

1 

10 

Hardware 

1 

2 

2 

Software 

Conceptual  Simplicity 

1 

2 

2 

Use 

6 

2 

12 

Training 

System  Resources 

2 

3 

6 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

6 

3 

18 

133.0 
x  3.4 
452.2 
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CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


o  IHPLEHENTATION:  FORTH AN  77 
o  FOB  CONCEPT:  #2  High-Order  Language 
o  SATISFIES  NEED:  *57  Software  Development  Tools 


.  EVALUATION  CRITERIA 

.  EVALUATION  X  HEIGHT 

=  SCORE 

Support  Documentation 
Diagnostics 

8 

2 

16 

Docuaentation 

6 

2 

12 

Interactive  Support 

4 

2 

8 

flaturity 

5 

3 

15 

Availability 

10 

3 

30 

Hardware  Coapatibility 

9 

3 

27 

Environaent  Coapatibility 

10 

3 

30 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

8 

2 

16 

Software 

Conceptual  Simplicity 

9 

2 

18 

Use 

10 

2 

20 

Training 

systea  Resources 

9 

3 

27 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

7 

3 

21 

250.0 
x  4.2 

1050.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION :  ADA 
o  FOR  CONCEPT:  #2  High-Order  Language 
o  SATISFIES  NEED:  *57  Software  Developaent  Tools 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Support  Docuaentation 
Diagnostics 

10 

2 

20 

Docuaentation 

10 

2 

20 

Interactive  Support 

8 

2 

16 

Haturity 

1 

3 

3 

Availability 

2 

3 

6 

Hardware  Compatibility 

3 

3 

9 

Environaent  Compatibility 

3 

3 

9 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

1 

2 

2 

Software 

Conceptual  Siaplicity 

1 

2 

2 

Use 

6 

2 

12 

Training 

System  Resources 

2 

3 

6 

Allocations  Reguired 

TOTAL 

NEED  WEIGHT 

CIB  SCORE 

6 

3 

18 

133.0 
x  4.2 
558.6 

293 
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CONCEPT  IHPLEHEBTATION  EVALUATION  SHEET 


O  IHPLBHENTATION:  dual 
o  FOB  CONCEPT:  12  High-Order  Language 
o  SATISFIES  NEED:  *57  Software  Development  Tools 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

*  SCORE 

Support  Documentation 
Diagnostics 

0 

2 

0 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

o 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Environeent  Compatibility 

9 

3 

27 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

10 

2 

20 

Software 

Conceptual  Simplicity 

10 

2 

20 

Use 

5 

2 

10 

Training 

Systea  Resources 

0 

3 

0 

Allocations  Beguired 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

5 

3 

15 

187.0 
X  4.2 
785.4 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  I HPL EH ENT ATI ON:  ONIVAC  1100/62  -  UNADS 
o  FOB  CONCEPT:  *3  Single  Large  Hulti-Usec  Env. 
o  SATISFIES  NEED:  *11  Decreased  Paperwork 

EVALUATION  CRITEHI A  .  EVALUATION  X  HEIGHT  *  SCOBE 


Interactive  Capability 
Maturity 
Vendor  Support 
Availability 
Hardware  Coapatibility 
Environment  Coapatibility 
Flexibility  of  Use 
Software 

Conceptual  Simplicity 
Use 

Training 

System  Resources 

Capabilities  Supported 


TOTAL 

NEED  HEIGHT 

CIB  SCORE 


176.0 
X  2.0 

352.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IHPLBHENTATION:  UNIVAC  1100/62  -  4000  TEftHINALS 
o  FOB  CONCEPT:  *3  Single  Large  Hnlti-User  Env . 
o  SATISFIES  NEED:  *55  Hodern  Source  Data  Entry  Tech's 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGBT 

»  SCORE 

Interactive  Capability 

io 

3 

30 

Maturity 

6 

3 

18 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coepatibility 

10 

3 

30 

Environeent  Coepatibility 
Flexibility  of  Use 

10 

3 

30 

Software 

7 

2 

14 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Systea  Resources 

Capabilities  Supported 

7 

3 

21 

TOTAL 

222.0 

NEED  HEIGHT 

X  4.6 

CIE  SCORE 

1021.2 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION :  UNIVAC  1100/62 
o  FOB  CONCEPT:  *3  Single  Large  Multi-User  Env. 
o  SATISFIES  NEED:  *60  Standardized  Development  Hardware 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Maturity 

6 

3 

18 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Conpatibility 

10 

3 

30 

Environment  Compatibility 
Flexibility  of  Use 

10 

3 

30 

Software 

7 

2 

1U 

Conceptual  Simplicity 

Use 

e 

2 

16 

Training 

8 

3 

24 

System  Resources 

Capabilities  Supported 

7 

3 

21 

TOTAL 

222.0 

NEED  HEIGHT 

x  3.8 

CIE  SCOBE 

843.6 

CONCEPT  IB PRESENTATION  EVALUATION  SHEET 


o  IBPLEBENTATION:  VA1  -  IS/1 

o  TOR  CONCEPT:  14  Standard  Small  Multiple  Env's 
o  SATISFIES  NEED:  *3  Interactive  System  Access 

EVALUATION  CRITERIA  -  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

10 

3 

30 

Maturity 

9 

3 

27 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

3 

3 

9 

Environment  Compatibility 
Flexibility  of  Use 

4 

3 

12 

Software 

Conceptual  Simplicity 

5 

2 

10 

Use 

8 

2 

16 

Training 

System  Resources 

8 

3 

24 

Capabilities  Supported 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

8 

3 

24 

191.0 
*  5.0 

955.0 
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CONCEPT  I HPLEHENTATI ON  EVALUATION  SHEET 


O  IHPLBHBNTATION:  SEL  -  SOFTOOL  80 
o  FOB  CONCEPT:  #4  Standard  Saall  Multiple  Env's 
o  SATISFIES  NEED:  *3  Interactive  Systea  Access 


EVALUATION  CBITEHIA  .  EVALUATION  X  HEIGHT  *  SCOBS 


Interactive  Capability 

10 

3 

30 

Haturity 

10 

3 

30 

Vendor  Support 

8 

1 

8 

Availability 

10 

3 

30 

Hardware  Coapatibility 

1 

3 

3 

Environaent  Coapatibility 
Flexibility  of  Use 

4 

3 

12 

Software 

5 

2 

10 

Conceptual  Siaplicity 

Use 

6 

2 

12 

Training 

6 

3 

18 

Systea  Besources 

Capabilities  Supported 

7 

3 

21 

TOTAL 

174.0 

NEED  HEIGHT 

x  5.0 

CIE  SCOBE 

870.0 
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CONCEPT  TUPLE BENT* TION  EVALUATION  SHEET 
o  IB PL EH ENT AT ION:  HABRIS  -  SES 

o  FOB  CONCEPT:  44  standard  saall  Multiple  Env's 
o  SATISFIES  NEED:  43  Interactive  Systea  Access 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

*  SCORE 

Interactive  Capability 

10 

3 

30 

Maturity 

8 

3 

24 

Vendor  Support 

10 

1 

10 

Availability 

10 

3 

30 

Hardware  Coapatibility 

1 

3 

3 

Environaent  Coapatibility 
Flexibility  of  Use 

5 

3 

15 

Software 

6 

2 

12 

Conceptual  Siapllcity 

Use 

7 

2 

14 

Training 

7 

3 

21 

systea  Resources 

Capabilities  Supported 

7 

3 

21 

TOTAL 

180.0 

NEBD  HEIGHT 

x  5.0 

CIE  SCORE 

900.0 
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Interactive  Capability 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environment  Compatibility 
Flexibility  of  Use 
Software 

Conceptual  Simplicity 
Use 

Training 

System  Resources 

Capabilities  Supported 


TOTAL 

HEED  HEIGHT 
Cl E  SCORE 


168.0 
X  5.0 
840. 0 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


!  o  IMPLEMENTATION:  VAX 

.  o  FOB  CONCEPT:  *4  Standard  Small  Multiple  Bnv's 

.  o  SATISFIES  NEED:  *4  Increased  Nueber  of  Terminals 

•  • 

.  EVALUATION  CRITERIA 

EVALUATION  X 

HEIGHT 

=  SCORE 

• 

Interactive  Capability 

8 

3 

24 

Maturity 

10 

3 

30 

Vendor  Support 

9 

1 

9 

Availability 

9 

3 

27 

Hardware  Compatibility 

7 

3 

21 

Environment  Compatibility 

8 

3 

24 

Flexibility  of  Use 

Software 

8 

2 

16 

Conceptual  Simplicity 

Use 

7 

2 

14 

Training 

7 

3 

21 

System  Resources 

Capabilities  Supported 

7 

3 

21 

TOTAL 

207.0 

NEED  HEIGHT 

X  4.6 

Cl E  SCORE 

952.2 

302 


* 


COHCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  SEL 

o  FOR  COHCEPT:  #4  Standard  Saall  Multiple  Env's 
o  SATISFIES  NEED:  *4  Increased  Nuaber  of  Terninals 


EVALUATION  CRITERIA 

.  EVALUATION  X  HEIGHT  *  SCORE 

Interactive  Capability 

8 

3 

24 

Maturity 

10 

3 

30 

Vendor  Support 

8 

1 

8 

Availability 

9 

3 

27 

Hardware  Coapatibility 

6 

3 

18 

Environment  Coapatibility 

6 

3 

18 

Flexibility  of  Use 

Software 

5 

2 

10 

Conceptual  Siaplicity 

Use 

7 

2 

14 

Training 

7 

3 

21 

Systea  Resources 

Capabilities  Supported 

5 

3 

15 

TOTAL 

185.0 

NEED  HEIGHT 

X  4.6 

CIE  SCORE 

851.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  HARRIS 

o  FOR  CONCEPT:  *4  Standard  SaalX  Multiple  Env's 
o  SATISFIES  NEED:  *4  Increased  Huaber  of  Terainals 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

9 

3 

27 

Maturity 

10 

3 

30 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coapatibility 

5 

3 

15 

Environaent  Coapatibility 

5 

3 

15 

Flexibility  of  Use 

software 

5 

2 

10 

Conceptual  siaplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Systea  Resources 

Capabilities  Supported 

6 

3 

18 

TOTAL 

194.0 

NEED  HEIGHT 

X  4.6 

Cl E  SCORE 

892.4 

304 
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CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


o  I H PLEA ENT AT ION:  POP  11/70 

o  FOB  CONCEPT:  *4  Standard  Stall  Hultiple  Env's 
o  SATISFIES  NEED:  *4  Increased  Nutber  of  Terninals 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  *  SCOBR 


Interactive  Capability 
Haturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environaent  Coapatibi lity 
Flexibility  of  use 
Software 

Conceptual  Siaplicity 
Use 

Training 

Systea  Resources 

Capabilities  Supported 

TOTAL 

NEED  HEIGHT 
Cl E  SCORE 


3 

3 

1 

3 

3 

3 

2 

2 

3 


24 

30 

9 

27 

24 

27 

12 

14 
21 

15 

203.0 
X  4.6 
933.8 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IB PL SB EH TAT ION:  SEL 

o  FOB  CONCEPT:  *4  standard  Stall  Multiple  Env's 
o  SATISFIES  NEED:  136  Graphics  Aids 


EVALUATION  CBITEBIA 

.  EVALUATION 

X  HEIGHT 

=  SCOBE 

Interactive  Capability 

10 

3 

30 

Baturity 

9 

3 

27 

Vendor  Support 

B 

1 

8 

Availability 

10 

3 

30 

Hardware  Coapatibility 

7 

3 

21 

Environment  Coapatibility 

7 

3 

21 

Flexibility  of  Use 

Software 

8 

2 

16 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

7 

3 

21 

Systea  Besources 

Capabilities  Supported 

8 

3 

24 

TOTAL 

NEED  HEIGHT 
CIE  SCOBE 


214.0 
I  3.4 
727.6 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IBPLEHENTATION:  TAX 

o  FOR  CONCEPT:  *4  Standard  Small  Multiple  Env's 
o  SATISFIES  NEED:  *36  Graphics  Aids 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  10 

Maturity  9 

Vendor  Support  8 

Availability  10 

Hardware  Compatibility  7 

Environaent  Compatibility  7 

Flexibility  of  Use 

Software  8 

Conceptual  Simplicity 

Use  8 

Training  7 

System  Resources 

Capabilities  Supported  8 


3 

3 

1 

3 

3 

3 


30 

27 

8 

30 

21 

21 


2  16 


2  16 

3  21 


3  24 


214.0 
X  3.4 
727.6 


.> .  »r  —  Oil' 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


o  Id PL EH EN TAT ION :  HARRIS 

o  FOB  CONCEPT:  *4  Standard  Saall  Hultiple  Env's 
o  SATISFIES  NEED:  *36  Graphics  Aids 


EVALUATION  criteria 


EVALUATION  X  HEIGHT  *  SCORE 


Interactive  Capability  5 

flaturity  8 

Vendor  Support  10 

Availability  10 

Hardware  Compatibility  7 

Environment  Compatibility  1 

Flexibility  of  Use 

Software  9 

Conceptual  Simplicity 

Use  7 

Training  7 

System  Resources 

Capabilities  Supported  3 


3 

3 

1 

3 

3 

3 


15 

24 

10 

30 

21 

3 


2  18 


2  14 

3  21 


3  9 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


165.0 
X  3.4 
561.0 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  VAX 

o  FOB  CONCEPT:  #4  Standard  saall  Multiple  Env's 
o  SATISFIES  NEED:  #52  Decrease  Turnaround  Tiae 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

8 

3 

24 

Matur  ity 

10 

3 

30 

Vendor  Support 

9 

1 

9 

Availability 

9 

3 

27 

Hardware  Coapatibility 

7 

3 

21 

Environment  Coapatibility 

8 

3 

24 

Flexibility  of  Use 

Software 

8 

2 

16 

Conceptual  Sinplicity 

Use 

7 

2 

14 

Training 

7 

3 

21 

System  Resources 

Capabilities  Supported 

7 

3 

21 

TOTAL 

207.0 

NEED  WEIGHT 

x  3.4 

CIE  SCORE 

703.8 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IMPLEMENTATION:  HABBIS 

o  EOB  CONCEPT:  ttt  Standard  snail  Multiple  Env's 
o  SATISFIES  NBED:  *52  Decrease  Turnaround  Tine 

EVALUATION  CBITEBI A  .  EVALUATION  X  HEIGHT  =  SCOBB 


Interactive  Capability 

9 

3 

27 

Maturity 

10 

3 

30 

vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coapatibility 

5 

3 

15 

Environment  Coapatibility 

5 

3 

15 

Flexibility  of  Use 

Software 

5 

2 

10 

Conceptual  Simplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Systea  Resources 

Capabilities  Supported 

6 

3 

18 

TOTAL 

NEED  HEIGHT 
CIS  SCORE 


194.0 
X  3.4 
659.6 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IHPLEHENTITION:  POP  11/70 


.  o  FOB  CONCEPT:  *4  standard  Saall  Multiple 

.  o  SATISFIES  NEED:  *52  Decrease  Turnaround 

Env's 

Tine 

.  EVALOATION  CRITERIA 

.  EVALOATION  Z  WEIGHT 

=  SCORE 

Interactive  Capability 

8 

3 

24 

Maturity 

10 

3 

30 

Vendor  Support 

9 

1 

9 

Availability 

9 

3 

27 

Hardv^re  Coapatibility 

8 

3 

24 

Enviicnaent  Coapatibility 

9 

3 

27 

Flexibility  of  Use 

Software 

6 

2 

12 

Conceptual  Siaplicity 

Use 

7 

2 

14 

Training 

7 

3 

21 

Systea  Resources 

Capabilities  Supported 

5 

3 

15 

TOTAL 

203.0 

NEED  WEIGHT 

x  3.4 

C IE  SCORE 

690.2 

311 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  SEL 

o  FOB  CONCEPT:  #4  standard  snail  Multiple  Env's 
o  SATISFIES  NEED:  *52  Decrease  Turnaround  Tiae 


EVALUATION  CRITERIA 

• 

EVALUATION  I  HEIGHT  * 

SCOBE 

Interactive  Capability 

8 

3 

24 

Maturity 

10 

3 

30 

Vendor  Support 

8 

1 

6 

Availability 

9 

3 

27 

Hardware  Coapatibility 

6 

3 

18 

Environaent  Coapatibility 
Flexibility  of  Use 

6 

3 

18 

Software 

Conceptual  Siaplicity 

5 

2 

10 

Use 

7 

2 

14 

Training 

Systea  Resources 

7 

3 

21 

Capabilities  Supported 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

5 

3 

15 

185.0 
x  3.4 
629.0 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  I HPLEHENTATION:  IS/1 

o  FOB  CONCEPT:  *4  standard  Saall  Kultiple  Env's 
o  SATISFIES  NEED:  *55  Modern  Source  Data  Entry  Tech's 

EVALUATION  CBITEBIA  .  EVALUATION  Z  HEIGHT  *  SCOBE 


Interactive  Capability 

10 

3 

30 

Maturity 

9 

3 

27 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

3 

3 

9 

Environment  Compatibility 
Flexibility  of  Use 

6 

3 

18 

Software 

7 

2 

1U 

Conceptual  Simplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

System  Resources 

Capabilities  Supported 

7 

3 

21 

TOTAL 

198.0 

NEED  HEIGHT 

x  4.6 

CIE  SCOBE 

910.8 

313 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


O  IHPLEHENTATION:  ORIX 

o  FOB  CONCEPT:  *4  standard  Small  Rultiple  Bnv's 
o  SATISFIES  NEED:  »55  Hodern  Source  Data  Entry  Tech's 


EVAL0ATION  CRITERIA 


Interactive  Capability 
flatur ity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environment  Compatibility 
Flexibility  of  Dse 
Software 

Conceptual  Simplicity 
ose 

Training 

System  Resources 

Capabilities  Supported 


EVALUATION  X  HEIGHT  = 


10  3 

10  3 

5  1 

10  3 

8  3 

3  3 

10  2 

1  2 

1  3 

7  3 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


CONCEPT  I HP L EBERT ATI 01  EVALUATION  SHEET 
o  IBPLEflENTATIOV:  SBS 

o  FOR  CONCEPT:  14  standard  Saall  Bultiple  Env'a 
o  SATISFIES  NEED:  #55  Bodern  Source  Data  Entry  Tech's 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Maturity 

8 

3 

24 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coapatibility 

2 

3 

6 

Environeent  Coapatibility 
Flexibility  of  Use 

5 

3 

15 

Software 

5 

2 

10 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Systea  Resources 

Capabilities  Supported 

6 

3 

18 

TOTAL 

182.0 

NEED  HEIGHT 

X  4.6 

CIE  SCORE 

837.2 

315 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 

O  IMPLEMENTATION:  VAX 

o  FOE  CONCEPT:  #4  standard  Small  Multiple  Env's 

o  SATISFIES  NEED:  *60  Standardized  Development  Hardware 

EVALUATION  CRITEBIA 

EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

8 

3 

24 

Maturity 

10 

3 

30 

Vendor  Support 

9 

1 

9 

Availability 

9 

3 

27 

Hardware  Compatibility 

7 

3 

21 

Environment  Compatibility 
Flexibility  of  Use 

8 

3 

24 

Software 

Conceptual  Simplicity 

8 

2 

16 

Use 

7 

2 

14 

Training 

System  Resources 

7 

3 

21 

Capabilities  Supported 

7 

3 

21 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


207.0 
X  3.8 
786.6 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IMPLEMENTATION:  SSL 

o  FOB  CONCEPT:  *4  standard  SaaXX  Multiple  Env's 
o  SATISFIES  NEED:  (60  Standardized  Development  Hardware 


EVALUATION  CRITEHI A 

.  EVALUATION  X  HEIGHT 

=  SCORE 

• 

Interactive  Capability 

8 

3 

24 

Maturity 

10 

3 

30 

Vendor  Support 

8 

1 

8 

Availability 

9 

3 

27 

Hardware  Compatibility 

6 

3 

18 

Environment  Compatibility 
Flexibility  of  Use 

6 

3 

18 

Software 

5 

2 

10 

Conceptual  Simplicity 

Use 

7 

2 

14 

Training 

7 

3 

21 

System  Resources 

Capabilities  Supported 

5 

3 

15 

TOTAL 

185.0 

NEED  HEIGHT 

x  3.8 

CIE  SCORE 

703.0 

317 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
O  IM PLED ENT AT ION:  HARRIS 

o  FOB  CONCEPT:  tu  Standard  Snail  Multiple  Euv's 
o  SATISFIES  NEED:  160  Standardized  Development  Hardware 


.  EVALUATION  CRITEBI A 

.  EVALUATION  X  HEIGHT  =  SCORE 

Interactive  Capability 

9 

3 

27 

Haturity 

10 

3 

30 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

5 

3 

15 

Flexibility  of  Ose 

Software 

5 

2 

10 

Conceptual  Simplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

System  Resources 

Capabilities  Supported 

6 

3 

10 

TOTAL 

194.0 

NEED  HEIGHT 

x  3.8 

CIE  SCORE 

737.2 

318 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  SCCS 

o  P08  CONCEPT:  *5  Configuration  Control  System 
o  SATISFIES  NEED:  »9  Configuration  Control 


EVALUATION  CRITEBIA 

.  EVALUATION 

X  WEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Docuaentation 
Diagnostics 

7 

2 

14 

Docuaentation 

3 

2 

6 

Interactive  Support 

5 

2 

10 

Autoaated  Procedure 

10 

2 

20 

Maturity 

9 

3 

27 

Availability 

10 

3 

30 

Hardware  coapatibility 

3 

3 

9 

Environment  Coapatibility 

5 

3 

15 

Governaent  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

Software 

Conceptual  Simplicity 

7 

2 

14 

Use 

7 

2 

14 

Training 

Systea  Resources 

9 

3 

27 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

C IE  SCORE 

5 

3 

15 

242.0 
X  4.0 
968.0 

CONCEPT  IHPLEHBNTATION  EVALUATION  SHEET 


o  IBPLEHBRTATIDR:  CCS 

o  FOB  CONCEPT:  #5  Configuration  control  Systea 
o  SATISFIES  NEED:  *9  configuration  Control 


• 

• 

a 

.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

* 

Interactive  Capability 

5 

3 

15 

Support  Docuaentation 

7 

2 

1U 

Diagnostics 

Docuaentation 

3 

2 

6 

Interactive  Support 

5 

2 

10 

Automated  Procedure 

10 

2 

20 

Haturity 

10 

3 

30 

Availability 

5 

3 

15 

Hardware  Compatibility 

3 

3 

9 

Environaent  Compatibility 

5 

3 

15 

Government  Access 

5 

1 

5 

S  Flexibility  of  Use 

Hardware 

3 

2 

6 

I  Software 

7 

2 

14 

I  Conceptual  Siaplicity 

1  Ose 

7 

2 

14 

|  Training 

9 

3 

27 

1  System  Resources 

I  Allocations  Required 

5 

3 

15 

i 

1  TOTAL 

215.0 

i 

1  NEED  HEIGHT 

x  4.0 

1  CIE  SCORE 

860.0 

V 

4 

320 

!| 

♦ 

. 

CONCEPT  IN PLEA ENT ATI ON  EVALUATION  SHEET 
o  IHPLEHENTATION:  slib 

o  POB  CONCEPT:  15  Configuration  Control  System 
o  SATISFIES  NEED:  #9  Configuration  Control 


.  EVALUATION  CRITERIA 

EVALUATION  X  WEIGHT 

*  SCORE 

* 

Interactive  Capability 

8 

3 

24 

Support  Documentation 
Diagnostics 

0 

2 

0 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

iutoaated  Procedure 

7 

2 

14 

Maturity 

7 

3 

21 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

5 

3 

15 

Government  Access 

Flexibility  of  Use 

8 

1 

8 

Hardware 

7 

2 

14 

Software 

10 

2 

20 

Conceptual  Simplicity 

Use 

5 

2 

10 

Training 

0 

3 

0 

System  Resources 

Allocations  Required 

5 

3 

15 

TOTAL 

186.0 

NEED  WEIGHT 

X  4.0 

CIS  SCORE 

744.0 

CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


O  IBPLBBEHTATION:  SBS 

o  FOB  CONCEPT:  »5  Configuration  Control  Systea 
o  SATISFIES  NEED:  *9  Configuration  Control 

EVALUATION  CRITERIA  .  EVALUATION  X  WEIGHT  =  SCORE 


Interactive  Capability 
Support  Docuaentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Autoaated  Procedure 
Baturity 
Availability 
Hardware  Cospatibility 
Environaent  Coapatibility 
Governaent  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Siaplicity 
Use 

Training 

Systea  Resources 

Allocations  Required 


TOTAL 

NEED  WEIGHT 
CIE  SCORE 


116.0 
X  4.0 
964.0 


CONCEPT  IHPLERENTATION  EVALUATION  SHEET 
o  I HPL EBERT AT ION:  SPAS 

o  POE  CONCEPT:  #5  Configuration  Control  System 
o  SATISFIES  NEED:  #9  Configuration  Control 


EVALUATION  CRITERIA 

.  EVALUATION  X 

HEIGHT  - 

SCORE 

Interactive  Capability 

io 

3 

30 

Support  Documentation 
Diagnostics 

5 

2 

10 

Documentation 

5 

2 

10 

Interactive  Support 

5 

2 

10 

Automated  Procedure 

5 

2 

10 

Haturity 

3 

3 

9 

Availability 

10 

3 

30 

Hardware  Compatibility 

1 

3 

3 

Environment  Compatibility 

1 

3 

3 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

1 

2 

2 

Software 

Conceptual  Simplicity 

10 

2 

20 

Ose 

8 

2 

16 

Training 

System  Resources 

8 

3 

24 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIS  SCORE 

8 

3 

2<* 

206.0 
I  4.0 
824.0 

323 


CONCEPT  IRPLEHENTATION  EVALUATION  SHEET 


o  IRPLEHENTATION:  SOFTOOL  II 

o  FOB  CONCEPT:  *5  Configuration  Control  Systea 
o  SATISFIES  NEED:  *9  Configuration  Control 

EVALOATION  CRITERIA  .  EVALUATION  X  HEIGHT  *  SCORE 


Interactive  Capability 
Support  Docuaentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Autoaated  Procedure 
Raturity 
Availability 
Hardware  Compatibility 
Environaent  Coapatibil ity 
Governaent  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Siaplicity 
Use 

Training 

Systea  Resources 

Allocations  Required 


10  3  30 

3  2  6 


2 

0 

0 

1 

6 

5 

U 

5 


2 

2 

2 

3 

3 

3 

3 

1 


4 
0 
0 
3 

18 

15 

12 

5 


7  2  14 

7  2  14 


6  2  12 

8  3  24 


0 


3  0 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


157.0 
X  4.0 
628.0 


3  24 


4 


CONCEPT  I BP LEH EH TATI ON  EVALUATION  SHEET 


o  I H PL BB BN TAT I ON:  SOFTOOL  II 

o  fob  CONCEPT:  AS  Configuration  Control  System 
o  SATISFIES  NEED:  *41  Organ.  Tools/Technigues  Interface 


EVALUATION  CRITERIA 


EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  10 

Support  Documentation  3 

Diagnostics 

Documentation  2 

Interactive  Support  0 

Automated  Procedure  0 

Haturity  1 

Availability  6 

Hardware  Compatibility  5 

Environment  Compatibility  4 

Government  Access  5 

Flexibility  of  Use 

Hardware  7 

Software  7 

Conceptual  Simplicity 

Use  6 

Training  8 

System  Resouices 

Allocations  Reguired  0 


3 

2 

2 

2 

2 

3 

3 

3 

3 

1 

2 

2 

2 

3 

3 


30 

6 

4 
0 
0 
3 

18 

15 

12 

5 

14 

14 

12 

24 

0 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


157.0 
X  3.4 
533.8 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION :  SCCS 

o  FOB  CONCEPT:  #5  Configuration  Control  Systea 
o  SATISFIES  NEED:  *42  User  Assistance  Function 


.  EVALUATION  CRITERIA 

.  EVALUATION  X 

HEIGHT 

*  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Docuaentation 
Diagnostics 

7 

2 

14 

Docuaentation 

3 

2 

6 

Interactive  Support 

5 

2 

10 

Autoaated  Procedure 

10 

2 

20 

Maturity 

9 

3 

27 

Availability 

10 

3 

30 

Hardware  Coapatibility 

3 

3 

9 

Environaent  Coapatibility 

5 

3 

15 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

Software 

Conceptual  Siaplicity 

7 

2 

14 

Use 

7 

2 

14 

Training 

Systea  Resources 

9 

3 

27 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

5 

3 

15 

242.0 
x  3.6 
871.2 

1 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  CCS 

o  FOR  CONCEPT:  #5  Configuration  Control  Systea 
o  SATISFIES  NEED:  #42  User  Assistance  Function 


EVALUATION  CRITERIA  .  EVALUATION  I  HEIGHT  =  SCORE 


Interactive  Capability 

5 

3 

Support  Docuaentation 
Diagnostics 

7 

2 

14 

Docuaentation 

3 

2 

6 

Interactive  Support 

5 

2 

10 

Autoaated  Procedure 

10 

2 

20 

Maturity 

10 

3 

30 

Availability 

5 

3 

15 

Hardware  Coapatibility 

3 

3 

9 

Environaent  Coapatibility 

5 

3 

15 

Governaent  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

Software 

Conceptual  Siaplicity 

7 

2 

14 

Use 

7 

2 

14 

Training 

Systea  Resources 

9 

3 

27 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

5 

3 

15 

215.0 
x  3.6 
774.0 

327 


ft 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  SLIB 

o  FOB  CONCEPT:  15  Configuration  Control  Systea 
o  SATISFIES  NEED:  *42  User  Assistance  Function 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  -  SCOBE 


Interactive  Capability 
Support  Docuaentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Autonated  Procedure 
Maturity 
Availability 
Hardware  Compatibility 
Environaent  Compatibility 
Governaent  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Siaplicity 
Use 

Training 

Systea  Resources 

Allocations  Heguired 


8  3  24 

0  2  0 


0 

0 

7 

7 

10 

5 

5 

8 


2 

2 

2 

3 

3 

3 

3 

1 


0 

0 

14 
21 
30 

15 
15 

8 


7  2  14 

10  2  20 

5  2  10 

0  3  0 


5  3  15 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


186.0 
X  3.6 
669.6 


328 


i 


Interact! 
Support  D 
Diagnosti 
Docume 
Intera 
Automated 
Maturity 
Availabil 
Hardware 
Environme 
Governmen 
Flexibili 
Hardwa 
Sof twa 
Conceptua 
Use 

Train! 
System  He 
A  1  loca 


ve  Capability 

ocuaentation 

cs 

ntation 
ctive  Support 
Procedure 

ity 

Compatibility 

nt  Compatibility 

t  Access 

ty  of  Use 

re 

re 

1  Simplicity 


tions  Required 


TOTAL 

NEED  HEIGHT 

CIE  SCORE 


116.0 
i  3.6 
U17.6 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IMPLEMENTATION:  SPMS 

o  FOB  CONCEPT:  *5  Configuration  Control  Systea 
o  SATISFIES  NEED:  *42  User  Assistance  Function 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

30 

Support  Docua- station 

5 

2 

10 

Diagnostics 

Docuaen tat  ion 

5 

2 

10 

Interactive  Support 

5 

2 

10 

Autoaated  Procedure 

5 

2 

10 

Maturity 

3 

3 

9 

Availability 

10 

3 

30 

Harduare  Coapatibility 

1 

i 

3 

Environaent  Coapatibility 

1 

3 

3 

Governaent  Access 

5 

1 

5 

Flexibility  of  Use 

Harduare 

1 

2 

2 

Software 

10 

2 

20 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Systea  Resources 

Allocations  Required 

8 

3 

24 

TOTAL 

206.0 

NEED  HEIGHT 

x  3.6 

CIE  SCORE 

741.6 

330 


't 


1 


CONCEPT  IHPLEHEMTATION  EVALUATION  SHEET 
O  IHPLEHENTATION:  IS/1  INmail 
o  FOB  CONCEPT:  »6  Automated  Office 
o  SATISFIES  NEED:  All  Decreased  Paperwork 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 
Automated  Procedure 
Availability 
Hardware  Compatibility 
Environment  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 
Software 

conceptual  Simplicity 
Use 

Training 


10  3  30 

10  2  20 

10  3  30 

3  3  9 

4  3  12 

5  1  5 

3  2  6 

1  2  2 

9  2  10 

9  3  27 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


159.0 

X  2.0 

318.0 


332 


i 


CONCEPT  IB PRESENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  IS/1  INed 
o  to 8  CONCEPT:  #6  Autoaated  Office 

o  SATISFIES  NEED:  *34  Autoaated  Text  Hanageaent  Systea 

EVALOATION  CHITB8IA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

10 

3 

30 

Automated  Procedure 

9 

2 

18 

Availability 

10 

3 

30 

Hardware  Compatibility 

3 

3 

9 

Environment  Coapatibility 

7 

3 

21 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

Software 

Conceptual  Siaplicity 

8 

2 

16 

Use 

7 

2 

14 

Training 

TOTAL 

NEED  HEIGHT 

CIS  SCORE 

4 

3 

12 

161.0 
x  3.8 
611.8 

CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 


O  IHPLEHENTATION:  OPTIHA 

o  FOR  CONCEPT:  17  Project  Hanagement  System 
o  SATISFIES  NEED:  *10  Improved  Hilestone  Identification 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

»  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

7 

2 

14 

Diagnostics 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Automated  Procedure 

5 

2 

10 

Haturity 

9 

3 

27 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Governaent  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

5 

2 

10 

Conceptual  Simplicity 

Use 

4 

2 

8 

Training 

4 

3 

12 

Output 

DMA  Applicable 

5 

3 

15 

Understandable 

5 

2 

10 

System  Resources 

Allocations  Required 

5 

3 

15 

TOTAL 

221.0 

NEED  HEIGHT 

x  3.0 

C IB  SCORE 

663.0 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  OPTIMA 

o  FOR  CONCEPT:  t 7  Project  Management  System 
o  SATISFIES  NEED:  Nil  Decreased  Paperwork 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

10 

3 

30 

Support  Documentation 
Diagnostics 

7 

2 

14 

Documentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Automated  Procedure 

5 

2 

10 

Maturity 

9 

3 

27 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

Conceptual  Simplicity 

5 

2 

10 

Use 

4 

2 

8 

Training 

Output 

4 

3 

12 

DMA  Applicable 

5 

3 

15 

Understandable 

Systea  Resources 

5 

2 

10 

Allocations  Required 

TOTAL 

SEED  HEIGHT 

CIB  SCORE 

5 

3 

15 

221.0 
x  2.0 
442.0 

335 


CONCEPT  IHPLEI1ERTATIOH  EVALUATION  SHEET 


O  IHPLEHENTATIOH:  BDP  1100 

o  FOR  CONCEPT:  *7  Project  Management  Systei 
o  SATISFIES  REED:  *11  Decreased  Paperwork 

EVALUATION  CRITERIA  .  EVALUATION  Z  HEIGHT  -  SCORE 


Interactive  Capability 

10 

3 

30 

Support  Docueentation 
Diagnostics 

0 

2 

0 

Docueentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoeated  Procedure 

4 

2 

e 

Maturity 

1 

3 

3 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

Conceptual  Siaplicity 

5 

2 

10 

Use 

0 

2 

0 

Training 

Output 

5 

3 

15 

DBA  Applicable 

5 

3 

15 

Understandable 

Systea  Resources 

0 

2 

0 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

5 

3 

15 

161.0 
x  2.0 
322.0 

CONCEPT  In PL MENTATION  EVALUATION  SHEET 
o  IH PL EH BN TAT ION:  CPAT 

o  PON  CONCEPT:  » 7  Project  Management  Systea 
o  SATISFIES  NEED:  All  Decreased  Paperwork 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

5 

3 

15 

Support  Documentation 
Diagnostics 

5 

2 

10 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Automated  Procedure 

5 

2 

10 

Haturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

Conceptual  Simplicity 

1 

2 

2 

Use 

5 

2 

10 

'"'■aining 

Output 

1 

3 

3 

DHA  Applicable 

8 

3 

24 

Understandable 

Systea  Resources 

8 

2 

16 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

Cl E  SCORE 

5 

3 

15 

205.0 
x  2.0 
410.0 

337 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  SCEBT  II 
o  FOB  CONCEPT:  #7  Project  Hanageaent  Systea 
o  SATISFIES  NEED:  til  Decreased  Paperwork 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

1 

3 

3 

Support  Docunentation 

8 

2 

16 

Diagnostics 

Docueentation 

1 

2 

2 

Interactive  Support 

1 

2 

2 

Autoaated  Procedure 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Coapatibility 

9 

3 

27 

Governaent  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

10 

2 

20 

Conceptual  Sieplicity 

Use 

8 

2 

16 

Training 

0 

3 

0 

Output 

DMA  Applicable 

5 

3 

15 

Understandable 

9 

2 

18 

Systea  Resources 

Allocations  Required 

0 

3 

0 

TOTAL 

NEED  HEIGHT 
CIR  SCORE 


199.0 
x  2.0 
388.0 


J 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  OPTIHA 

o  FOB  CONCEPT*.  *7  Project  Hanageeent  Systea 
o  SATISFIES  NEED:  112  Iaprove  flanloading 


.  EVALUATION  CRITERIA 

: 

EVALUATION 

X  WEIGHT 

=  SCORE 

. 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

7 

2 

14 

Diagnostics 

Docuaen tat  ion 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoeated  Procedure 

5 

2 

10 

Hatur ity 

9 

3 

27 

Availability 

10 

3 

30 

Hardware  coapatibility 

10 

3 

30 

Government  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

5 

2 

10 

Conceptual  Simplicity 

Use 

tt 

2 

8 

Training 

U 

3 

12 

Output 

DBA  Applicable 

5 

3 

15 

Understandable 

5 

2 

10 

Systea  Resources 

Allocations  Required 

5 

3 

15 

TOTAL 

221.0 

NEED  WEIGHT 

x  2.8 

Cl E  SCORE 

618.8 

339 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  PffICE 

o  FOB  CONCEPT:  B7  Project  Management  System 
o  SATISFIES  NEED:  172  laprove  Banloading 


EVALUATION  CRITEBIA 


EVALUATION  X  HEIGHT  =  SCOBS 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Availability 
Hardware  Compatibility 
Governaent  Access 
Flexibility  of  Use 
Hardware 

Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
Systea  Resources 

Allocations  Beguired 


10 

9 

1 

6 

7 

10 

10 

1 

3 


30 

18 

2 

12 

14 

30 

30 

3 

3 


10 

15 

12 

14 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


195.0 
x  2.8 
546.0 


340 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION :  SLIM 

o  FOB  CONCEPT:  *7  Project  Management  System 
o  SATISFIES  NEED:  *12  Improve  Hanloading 


EVALUATION  CHITBBIA  .  EVALUATION  Z  HEIGHT  =■  SCOBS 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 

Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
System  Besources 

Allocations  Beguired 


2  2 

7  2 

7  2 

9  3 

10  3 

0  3 

3  1 


1  2 


6  2 

4  3 


4  3 

6  2 


0  3 


30 

18 

4 

14 

14 

27 

30 

0 

3 

2 

12 

12 

12 

12 

0 


TOTAL 

NEED  HEIGHT 
CIS  SCOBE 


190.0 
x  2.8 
532.0 


341 


*3) 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  SCERT  II 

o  FOB  CONCEPT:  #7  Project  Management  System 
o  SATISFIES  NEED:  #12  Iaprove  Manloading 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  1 

Support  Docuaentation  8 

Diagnostics 

Docuaentation  1 

Interactive  Support  1 

Automated  Procedure  5 

Maturity  10 

Availability  10 

Hardware  Coapatibility  9 

Government  Access  5 

Flexibility  of  Use 

Hardware  10 

Conceptual  Simplicity 

Use  8 

Training  0 

Output 

DMA  Applicable  5 

Understandable  9 

system  Resources 

Allocations  Required  0 


3  3 

2  16 


2 

2 

2 

3 

3 

3 

1 


2 

2 

10 

30 

30 

27 

5 


2  20 


2  16 

3  0 


3  15 

2  18 


3  0 


TOTAL 

NEED  HEIGHT 
CIS  SCORE 


194.0 
x  2.8 
543.2 


342 


Interactive  Capability 
Support  Docuaentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Autoaated  Procedure 
Maturity 
Availability 
Hardware  Coapatibility 
Governaent  Access 
‘ lexibility  of  Dse 
Hardware 

Conceptual  Siaplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
Systea  Resources 

Allocations  Required 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


221.0 
x  2.6 
574.6 


••-ST-.:.. 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  PBICE 

o  FOR  CONCEPT:  *7  Project  Hanageaent  Systea 
o  SATISFIES  NEED:  VIA  Iaprove  Schedule  lapact  Analysis 


EVALUATION  CRITERIA  .  EVALUATION  *  WEIGHT  *  SCORE 


Interactive  Capability  10 

Support  Docuaentation  9 

Diagnostics 

Docuaentation  1 

Interactive  Support  6 

Autosated  Procedure  7 

Maturity  10 

Availability  10 

Hardware  Coapatibility  1 

Governnent  Access  3 

Flexibility  of  Use 

Hardware  1 

Conceptual  Siaplicity 

Use  5 

Training  5 

Output 

DMA  Applicable  4 

Understandable  7 

Systea  Resources 

Allocations  Required  0 


3  30 

2  18 


2 

2 

2 

3 

3 

3 

1 


2 

12 

14 

30 

30 

3 

3 


2  2 


10 

15 


12 

14 


TOTAL 

NEED  WEIGHT 
CIE  SCORE 


195.0 
x  2.6 
507.0 


344 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 

Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
System  Resources 

Allocations  Required 


TOTAL 

NEED  HEIGHT 
C IE  SCORE 


190.0 
i  2.6 
494.0 
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CONCEPT  IH PLEA ENT ATI ON  EVALUATION  SBBET 


O  INPLENENTATION:  SCERT  II 
o  FOB  CONCEPT:  <7  Project  Hanageaent  Systea 
o  SATISFIES  NEED:  410  Iaprove  Schedule  Impact  Analysis 


EVALUATION  CRITERIA 


EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  1 

Support  Docuaentation  8 

Diagnostics 

Docuaentation  1 

Interactive  Support  1 

Autoaated  Procedure  5 

Maturity  10 

Availability  10 

Hardware  Coapatibility  9 

Governaent  Access  5 

Flexibility  of  Use 

Hardware  10 

Conceptual  Siaplicity 

Use  8 

Training  0 

Output 

DBA  Applicable  5 

Understandable  9 

Systea  Resources 

Allocations  Required  0 


3 

2 

2 

2 

2 

3 

3 

3 

1 

2 

2 

3 

3 

2 

3 


3 

16 

2 

2 

10 

30 

30 

27 

5 

20 

16 

0 

15 

18 

0 


TOTAL 

NEED  HEIGHT 
CIS  SCORE 


194. 0 
x  2.6 
504. 4 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IMPLEMENTATION:  CPAT 

o  FOR  CONCEPT:  *7  Project  Management  Systea 
o  SATISFIES  REED:  *14  Improve  Schedule  Impact  Analysis 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

*  SCORE 

Interactive  Capability 

5 

3 

15 

Support  Docunentation 
Diagnostics 

5 

2 

10 

Documentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Automated  Procedure 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

Conceptual  Simplicity 

1 

2 

2 

Use 

5 

2 

10 

Training 

Output 

1 

3 

3 

DHA  Applicable 

8 

3 

24 

Understandable 

Systea  Resources 

8 

2 

16 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

5 

3 

15 

205.0 
x  2.6 
533.0 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  MAPPER 

o  EOS  CONCEPT:  *7  Project  Management  System 
o  SATISFIES  NEED:  140  Historical  Data  Base  Techniques 


EVALUATION  CRITERIA  .  EVALUATION  Z  HEIGHT  =  SCORE 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Availability 
Hardware  Conpatibility 
Governnent  Access 
Flexibility  of  Dse 
Hardware 

Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
System  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


348 


10 

3 

30 

9 

2 

18 

8 

2 

16 

8 

2 

16 

8 

2 

16 

8 

3 

24 

10 

3 

30 

10 

3 

30 

10 

1 

10 

5 

2 

10 

9 

2 

18 

8 

3 

24 

7 

3 

21 

8 

2 

16 

1 

3 

3 

282.0 
x  2.6 

233.2 

CONCEPT  INPLEHENTATION  EVALUATION  SHEET 


O  INPLEHENTATION :  OPTIMA 

o  POE  CONCEPT :  *7  Project  Management  systee 
o  SATISFIES  NEED:  *56  Hanageaent  Tracking  Functions 


.  EVALUATION  CRITERIA 

.  EVALUATION 

Z  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Docuaentation 
Diagnostics 

7 

2 

14 

Documentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Automated  Procedure 

5 

2 

10 

Maturity 

9 

3 

27 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Government  Access 
Flexibility  of  Use 

10 

1 

10 

Hardware 

Conceptual  Simplicity 

5 

2 

10 

Use 

4 

2 

8 

Training 

Output 

4 

3 

12 

DMA  Applicable 

5 

3 

15 

Understandable 

Systea  Resources 

5 

2 

10 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIS  SCORE 

5 

3 

15 

221.0 
x  3.2 
707.2 

CONCEPT  IHPLEHENTATIOH  EVALUATION  SHEET 


o  IHPLBHENTATION:  RDP  1100 

o  FOB  CONCEPT:  #7  Project  Hanageaent  Systee 
o  SATISFIES  NEED:  *56  Hanageaent  Tracking  Functions 


.  EVALUATION  CRITERIA 

.  EVALUATION 

I  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Docuaentation 

0 

2 

0 

Diagnostics 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

4 

2 

8 

Maturity 

1 

3 

3 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Governaent  Access 

5 

1 

5 

Flexibility  of  Ose 

Hardware 

5 

2 

10 

Conceptual  Siaplicity 

Use 

0 

2 

0 

Training 

5 

3 

15 

Output 

DHA  Applicable 

5 

3 

15 

Understandable 

0 

2 

0 

Systea  Resources 

Allocations  Required 

5 

3 

15 

TOTAL 

161.0 

NEED  HEIGHT 

x  3.2 

CIE  SCORE 

515.2 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  MAPPER 

o  FOB  CONCEPT:  B7  Project  Management  System 
o  SATISFIES  NEED:  IS6  Hanageaent  Tracking  Functions 


EVALUATION  CBITEBIA 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 

Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
System  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 
CIB  SCORE 


i 


EVALUATION  X  HEIGHT  =  SCORE 


10 

3 

30 

9 

2 

18 

8 

2 

16 

8 

2 

16 

8 

2 

16 

8 

3 

29 

10 

3 

30 

10 

3 

30 

10 

1 

10 

5 

2 

10 

9 

2 

18 

8 

3 

29 

7 

3 

21 

8 

2 

16 

1  3  3 

282.0 
x  3.2 
902.9 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
O  IHPLEHENTATION :  PRICE 

o  FOB  CONCEPT:  *7  Project  Hanageeent  systea 
o  SATISFIES  HEED:  *56  Hanageeent  Tracking  Functions 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  *  SCOBE 


30 

18 

2 

12 

14 
30 
30 

3 

3 

2 

10 

15 


12 

14 

0 

195.0 
*  3.2 

624.0 


i 
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Interactive  Capability  10  3 

Support  Docuaentation  9  2 

Diagnostics 

Docuaentation  1  2 

Interactive  Support  6  2 

Autoaated  Procedure  7  2 

Haturity  10  3 

Availability  10  3 

Hardware  Coapatibility  1  3 

Governaent  Access  3  1 

Flexibility  of  use 

Hardware  1  2 

Conceptual  Siaplicity 

Use  5  2 

Training  5  3 

Output 

DHA  Applicable  4  3 

Understandable  7  2 

Systea  Resources 

Allocations  Required  0  3 


TOTAL 

NEED  HEIGHT 
CIS  SCOBE 


1 


Interactive  Capability 
Support  Docuaentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 

Conceptual  Simplicity 
Ose 

Training 

Output 

DMA  Applicable 
Understandable 
System  Resources 

Allocations  Required 


TOTAL 

N  BED  HEIGHT 
CIE  SCORE 


190.0 
x  3.2 
608.0 


-J 


CONCEPT  IMPLEMENTATION  evaluation  SHEET 

o  18 PL EH ENT AT ION:  SCERT  II 

o  FOR  CONCEPT:  #7  Project  Management  Systea 

o  SATISFIES  NEED:  *56  Hanageaent  Tracking  Functions 

EVALUATION  CRITERIA  ! 

EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

1 

3 

3 

Support  Documentation 

Diagnostics 

8 

2 

16 

Documentation 

1 

2 

2 

Interactive  Support 

1 

2 

2 

Automated  Procedure 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Compatibility 

9 

3 

27 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

Conceptual  Simplicity 

10 

2 

20 

Use 

8 

2 

16 

Training 

Output 

0 

3 

0 

DMA  Applicable 

5 

3 

15 

Understandable 

System  Resources 

9 

2 

18 

allocations  Required 

TOTAL 

heed  height 

C IE  SCORE 

0 

3 

0 

19U.0 
x  3.2 
620.8 

354 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  SLIM 

o  PON  CONCEPT:  *8  Cost  Estimating  System 
o  SATISFIES  NEED:  *12  Isprove  Manloading 


EVALUATION  CRITERIA 


EVALUATION  l  HEIGHT  «  SCORE 


Interactive  Capability  10 

Support  Oocueentation  9 

Automated  Procedure  7 

Maturity  10 

Vendor  Support  9 

Availability  10 

Hardware  Cospatibility  0 

Government  Access  3 

Flexibility  of  Use 

Hardware  1 

Conceptual  Siaplicity 

Use  6 

Training  4 

Output 

DMA  Applicable  4 

Understandable  6 

System  Resources 

Allocations  Required  0 


3 

2 

2 

3 

1 

3 

3 

1 


30 

18 

14 

30 

9 

30 

0 

3 


2  2 


2  12 

3  12 


3  12 

2  12 

3  0 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


184.0 
X  2.8 
515.2 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  PHICE 

o  TOR  CONCEPT:  *8  Cost  Estimating  Systea 
o  SATISFIES  NEED:  *12  laprove  Manloading 


EVALUATION  CRITERIA 


EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

10 

3 

30 

Support  Docuaentation 

9 

2 

18 

Autoaated  Procedure 

7 

2 

14 

Maturity 

10 

3 

30 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coapatibility 

1 

3 

3 

Governaent  Access 

3 

1 

3 

Flexibility  of  Use 

Hardware 

1 

2 

2 

Conceptual  Siaplicity 

Use 

5 

2 

10 

Training 

5 

3 

15 

Output 

DMA  Applicable 

u 

3 

12 

Understandable 

7 

2 

14 

Systea  Resources 

j  Allocations  Required 

0 

3 

0 

i  TOTAL 

190.0 

;  NEED  HEIGHT 

x  2.8  ‘ 

CIE  SCORE 

1 

i 

\ 

i 

» 

532.0 

356 

] 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IBPLEHENTATION:  COCOMO 
o  FOB  CONCEPT:  *8  Cost  Estimating  Systea 
o  SATISFIES  NEED:  *12  Improve  Hanloading 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

i 

3 

3 

Support  Documentation 

9 

2 

18 

Automated  Procedure 

1 

2 

2 

Maturity 

1 

3 

3 

Vendor  Support 

1 

1 

1 

Availability 

10 

3 

30 

Hardware  Compatibility 

1 

3 

3 

Government  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

1 

2 

2 

Conceptual  Simplicity 

Use 

4 

2 

8 

Training 

1 

3 

3 

Output 

DMA  Applicable 

4 

3 

12 

Understandable 

5 

2 

10 

Systea  Resources 

Allocations  Required 

10 

3 

30 

TOTAL 

135.0 

NEED  HEIGHT 

x  2.8 

CIS  SCORE 

378.0 

357 


CONCEPT  IHPLEHEHTATION  EVALUATION  SHEET 
o  IBPLBHENTATION:  SLIB 

o  FOR  CONCEPT:  18  Cost  Estimating  System 
o  SATISFIES  NEED:  *1tt  Iaprove  Schedule  Impact  Analysis 


EVALUATION  CRITERIA 


Interactive  Capability 
Support  Documentation 
Automated  Procedure 
Maturity 
Vendor  Support 
Availability 
Hardware  Coapatibility 
Government  Access 
Flexibility  of  Use 
Hardware 

Conceptual  Simplicity 
Use 

Training 

Output 

DBA  Applicable 
Understandable 
System  Resources 

Allocations  Required 


TOTAL 

NEED  HEIGHT 
"IE  SCORE 


EVALUATION  X  HEIGHT  *  SCORE 


10 

9 

7 

10 

9 

10 

0 

3 


30 

18 

14 

30 

9 

30 

0 

3 


12 

12 

12 

12 


184.0 
x  2.6 
478.4 


358 


i 

1 


1 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  PRICE 
o  FOR  CONCEPT:  18  Cost  Estimating  Systes 
o  SATISFIES  NEED:  *14  Iapcove  Schedule  Impact  Analysis 


EVALDATION  CRITERIA  .  EVALDATION  I  HEIGHT  »  SCORE 


Interactive  Capability 
Support  Docuaentation 
Automated  Procedure 
Maturity 
Vendor  Support 
Availability 
Hardvare  Compatibility 
Government  Access 
Flexibility  of  Ose 
Hardvare 

Conceptual  Simplicity 
Use 

Training 

Output 


10 

9 

7 

10 

9 

10 

1 

3 


3 

2 

2 

3 

1 

3 

3 

1 


30 

18 

14 

30 

9 

30 

3 

3 


1  2  2 


5  2  10 

5  3  15 


DMA  Applicable 
Understandable 
System  Resources 

Allocations  Required 


4  3  12 

7  2  14 

0  3  0 


TOTAL 

NEED  HEIGHT 
CIS  SCORE 


190.0 
x  2.6 
494.0 


359 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  COCOHO 
o  FOR  CONCEPT :  18  Cost  Estimating  System 
o  SATISFIES  NEED:  #14  improve  Schedule  Iapact  Analysis 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  *  SCORE 


Interactive  Capability 

1 

3 

3 

Support  Docusentation 

9 

2 

18 

Autoaated  Procedure 

1 

2 

2 

Haturity 

1 

3 

3 

Vendor  Support 

1 

1 

1 

Availability 

10 

3 

30 

Hardware  Coapatibility 

1 

3 

3 

Governaent  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

Conceptual  Simplicity 

1 

2 

2 

Use 

4 

2 

8 

Training 

Output 

1 

3 

3 

DMA  Applicable 

4 

3 

12 

Understandable 

Systea  Resources 

5 

2 

10 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIB  SCORE 

10 

3 

30 

135.0 
x  2.6 
351.0 

360 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  SLIH 

o  FOB  CONCEPT:  *8  Cost  Estiaating  Systea 
o  SATISFIES  NEED:  *56  flanageaent  Tracking  Functions 


EVALUATION  CRITERIA 


EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  10 

Support  Docuaentation  9 

Autoaated  Procedure  7 

Maturity  10 

Vendor  Support  9 

availability  10 

Hardware  Coapatibility  0 

Governaent  Access  3 

Flexibility  of  Use 

Hardware  1 

Conceptual  Siaplicity 

Use  6 

Training  4 

Output 

DMA  Applicable  4 

Understandable  6 

Systea  Resources 

Allocations  Required  0 


3 

2 

2 

3 

1 

3 

3 

1 


30 

18 

14 

30 

9 

30 

0 

3 


2  2 

2  12 

3  12 


3  12 

2  12 

3  0 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


184.0 
x  3.2 
588.8 


361 


CONCEPT  IBPLE8ENTATI0N  EVALUATION  SHEET 


O  IBPLEBENTATION:  PRICE 

o  FOB  CONCEPT:  *8  Cost  Estiaating  System 
o  SATISFIES  NEED:  fS6  Hanageaent  Tracking  Functions 


EVALUATION  CRITERIA 


EVALUATION  1  HEIGHT  *  SCORE 


Interactive  Capability 
Support  Documentation 
Automated  Procedure 
Haturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 

Conceptual  Siaplicity 
Use 

Training 

Output 

DBA  Applicable 
Understandable 
Systea  Resources 

Allocations  Required 


10 

9 

7 

10 

9 

10 

1 

3 


3 

2 

2 

3 

1 

3 

3 

1 


30 

18 

14 

30 

9 

30 

3 

3 


1  2  2 


5  2  10 

5  3  15 


W  3  12 

7  2  14 


0 


3  0 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


190.0 
x  3.2 
608.0 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  COCOHO 
o  FOB  CONCEPT:  18  Cost  Estimating  Systea 
o  SATISFIES  NEED:  *56  Management  Tracking  Functions 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

■  SCORE 

Interactive  Capability 

1 

3 

3 

Support  Documentation 

9 

2 

18 

Automated  Procedure 

1 

2 

2 

Maturity 

1 

3 

3 

Vendor  Support 

1 

1 

1 

Availability 

10 

3 

30 

Hardware  Compatibility 

1 

3 

3 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

Conceptual  Simplicity 

1 

2 

2 

Use 

4 

2 

8 

Training 

Output 

1 

3 

3 

DMA  Applicable 

4 

3 

12 

Understandable 

System  Resources 

5 

2 

10 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIB  SCORE 

10 

3 

30 

135.0 
x  3.2 
432.0 

363 
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CONCEPT  1(1  PL  EH  BN  TATI  ON  EVALUATION  SHEET 


O  IH PL EH ENT AT ION:  PEBT 

o  FOB  CONCEPT:  *9  Project  Path  Analysis  Hethod 
o  SATISFIES  NEED:  *10  laptoved  Hilestone  Identification 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  *  SCOBE 


Interactive  Capability 

1 

3 

3 

Support  Docuaentation 

10 

2 

20 

Automated  procedure 

1 

2 

2 

Haturity 

10 

3 

30 

Availability 

10 

3 

30 

Environaent  Compatibility 

5 

3 

15 

Governaent  Access 

Conceptual  Siaplicity 

10 

1 

10 

Use 

5 

2 

10 

Training 

1 

3 

3 

Output 

OHA  Applicable 

8 

3 

24 

Understandable 

8 

2 

16 

TOTAL 

163.0 

NEED  HEIGHT 

*  3.0 

CIE  SCORE 

489.0 

364 


i 


Interactive  Capability 
Support  Docuaentation 
Autoaated  Procedure 
Maturity 
Availability 

Environaent  Coapatibility 
Governaent  Access 
Conceptual  Siaplicity 
Ose 

Training 

Output 

DMA  Applicable 
Understandable 


TOTAL 

NEED  HEIGHT 
CIS  SCOHE 


125.0 
x  3.0 
375.0 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  CPAT 

o  FOB  CONCEPT:  *9  Project  Path  Analysis  Hethod 
o  SATISFIES  NEED:  *10  Improved  Hilestone  Identification 


.  EVALUATION  CBITEBIA 

.  EVALUATION 

X  HEIGHT 

*  SCOBE 

Interactive  Capability 

5 

3 

15 

Support  Docuaentation 

5 

2 

10 

Autoaated  Procedure 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Environaent  coepatibility 

5 

3 

15 

Government  Access 

10 

1 

10 

Conceptual  Siaplicity 

Use 

5 

2 

10 

Training 

1 

3 

3 

Output 

DHA  Applicable 

8 

3 

24 

Understandable 

8 

2 

16 

TOTAL 

173.0 

NEED  HEIGHT 

I  3.0 

CIB  SCOBE 

519.0 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  PENT 

o  PON  CONCEPT:  *9  Project  Path  Analysis  Method 
o  SATISFIES  NEED:  *12  Iaprove  Ranloading 

EVALUATION  CRITERIA  .  EVALUATION  1  HEIGHT  *  SCORE 


Interactive  Capability 

1 

3 

3 

Support  Docuaentation 

10 

2 

20 

Autoaated  Procedure 

1 

2 

2 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Environaent  Compatibility 

5 

3 

15 

Governaent  Access 

Conceptual  Siaplicity 

10 

1 

10 

Use 

5 

2 

10 

Training 

Output 

1 

3 

3 

DMA  Applicable 

8 

3 

24 

Understandable 

TOTAL 

NEED  HEIGHT 

CIB  SCORE 

8 

2 

16 

163.0 
x  2.8 
456.4 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IHPLEHENTATION:  CPE 

o  FOR  CONCEPT:  *9  Project  Path  Analysis  Hethod 
o  SATISFIES  NEED:  *12  laprove  Hanloading 

EVALDATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 
Support  Documentation 
Automated  Procedure 
Maturity 
Availability 

Environment  compatibility 
Government  Access 
Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 


1  3  3 

8  2  16 

3  2  6 

10  3  30 

10  3  30 

5  3  15 

10  1  10 


0  2  0 

1  3  3 


«  3  12 

0  2  0 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


125.0 
X  2.8 
475.0 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  SCENT  II 

o  FOB  CONCEPT:  #9  Project  Path  Analysis  Method 
o  SATISFIES  NEED:  *12  Iaprove  Manloading 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

1 

3 

3 

Support  Docuaentation 

8 

2 

16 

Autoaated  Procedure 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Environaent  Coapatibility 

5 

3 

15 

Government  Access 

Conceptual  Siaplicity 

5 

1 

5 

Use 

8 

2 

16 

Training 

0 

3 

0 

Output 

DMA  Applicable 

5 

3 

15 

Understandable 

9 

2 

18 

TOTAL 

158.0 

NEED  HEIGHT 

x  2.8 

CIE  SCORE 

4H2.U 

369 


v 


CONCEPT  IH PL BEEN TATI ON 

EVALUATION 

SHEET 

o  IN  PL  E(1  ENT  AT  ION:  PEST 

o  EOS  CONCEPT:  »9  Project  Path 

Analysis 

Method 

o  SATISFIES  NEED:  *14  Ieprove 

Schedule  Iapact 

Analysis 

EVALOATION  CRITERIA 

• 

EVALUATION  X  HEIGHT 

*  SCORE 

Interactive  Capability 

1 

3 

3 

Support  Docuaentation 

10 

2 

20 

Autoaated  procedure 

1 

2 

2 

Naturity 

10 

3 

30 

Availability 

10 

3 

30 

Snvironaent  Coapatibility 

5 

3 

15 

Governaent  Access 

10 

1 

10 

Conceptual  Siaplicity 

Use 

5 

2 

10 

Training 

1 

3 

3 

Output 

DMA  Applicable 

8 

3 

24 

Understandable 

8 

2 

16 

TOTAL 

NEED  HEIGHT 
CIE  SCOBE 


163.0 
x  2.6 
423.8 


Interactive  Capability 
Support  Documentation 
Automated  Procedure 
Haturity 
Availability 

Environment  Compatibility 
Government  Access 
Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 


TOTAL 

NEED  WEIGHT 
CIB  SCOBE 


125.0 
z  2.6 
325.0 


CONCEPT  IMPLEMENTATION  EV ALOATION  SHEET 


o  IMPLEMENTATION:  CPAT 

o  FOB  CONCEPT:  *9  Project  Path  Analysis  Method 
o  SATISFIES  NEED:  #14  Improve  Schedule  Iapact  Analysis 


.  EVALUATION  CRITERIA 

.  EVALUATION  X 

HEIGHT  = 

SCORE 

Interactive  Capability 

5 

3 

15 

Support  Documentation 

5 

2 

10 

Autoaated  Procedure 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Environment  Coapatibility 

5 

3 

15 

Sovernaent  Access 

Conceptual  Simplicity 

10 

1 

10 

Use 

5 

2 

10 

Training 

Output 

1 

3 

3 

DBA  Applicable 

8 

3 

24 

Understandable 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

8 

2 

16 

173.0 
x  2.6 
449.8 

372 


CONCEPT  IHPLEHBNTATION  EVALUATION  SHEET 
o  IB PL EH ENT AT ION:  PEBT 

o  FOB  CONCEPT:  *9  Project  Path  Analysis  Bethod 
o  SATISFIES  NEED:  *56  Banageaent  Tracking  Functions 


EVALOATION  CRITERIA 

EVALUATION 

X  HEIGHT 

*  SCORE 

Interactive  Capability 

1 

3 

3 

Support  Documentation 

10 

2 

20 

Automated  Procedure 

1 

2 

2 

Baturity 

10 

3 

30 

Availability 

10 

3 

30 

Environaent  Coapatibi lity 

5 

3 

15 

Governaent  Access 

10 

1 

10 

Conceptual  Siaplicity 

Use 

5 

2 

10 

Training 

1 

3 

3 

Output 

DBA  Applicable 

8 

3 

2« 

Understandable 

8 

2 

16 

TOTAL 

163.0 

NEED  HEIGHT 

x  3.2 

CIE  SCORE 

521.6 

373 


CONCEPT  IB PL HABITATION  EVALUATION  SHEET 
o  IAPLEBBNTATION:  SCERT  II 

o  FOR  CONCEPT:  *9  Project  Path  Analysis  Method 
o  SATISFIES  NEED:  *56  Management  Tracking  Functions 


EVALUATION  CRITERIA 


Interactive  Capability 
Support  Docuaentation 
Automated  Procedure 
Maturity 
Availability 

Environaent  Coapat ibility 
Government  Access 
ConceDtual  Siaplicity 
Ose 

Training 

Output 

DBA  Applicable 
Understandable 


EVALUATION  X  HEIGHT  =  SCORE 


1 

8 

5 

10 

10 

5 

5 

8 

0 

5 

9 


3 

16 

10 

30 

30 

15 
5 

16 
0 

15 

18 


TOTAL 

NERD  HEIGHT 
CIE  SCORE 


158.0 
x  3.2 
505.6 


375 


WTJ  » 
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CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
O  IHPLEHENTATION:  CPAT 

o  FOB  CONCEPT:  *9  Project  Path  analysis  Bethod 
o  SATISFIES  NEED:  #56  Hanageaent  Tracking  Functions 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

5 

3 

15 

Support  Docuaentation 

5 

2 

10 

Autoaated  Procedure 

5 

2 

10 

Haturity 

10 

3 

30 

Availability 

10 

3 

30 

Environaent  Coapatibility 

5 

3 

15 

Governnent  Access 

10 

1 

10 

Conceptual  Siaplicity 

Use 

5 

2 

10 

Training 

1 

3 

3 

Output 

DBA  Applicable 

8 

3 

24 

Understandable 

8 

2 

16 

TOTAL 

173.0 

NEED  HEIGHT 

x  3.2 

CIE  SCORE 

553.6 

376 


CONCEPT  IBPLEHENTATION  EVALUATION  SHEET 
o  IHPLBHBNTATIOH:  HXPERTEXT 

o  FOR  CONCEPT:  #10  Software  Eng.  Practices  Training 
o  SATISFIES  NEED:  *18  Faster  Intergation  of  New  Eepl's 


• 

EVALUATION  CRITERIA 

* 

EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

1 

3 

3' 

Autoaated  Procedure 

10 

2 

20 

Haturity 

10 

3 

30 

Availability 

10 

3 

30 

Environment  Coapatibility 
Conceptual  Siaplicity 

B 

3 

24 

Use 

10 

2 

20 

TOTAL 

127.0 

NEED 

HEIGHT 

x  2.2 

CIB 

SCORE 

279.4 

377 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IHPLEflENTATION:  HYPERTEXT 

o  FOR  CONCEPT:  *10  software  Eng.  Practices  Training 
o  SATISFIES  NEED:  *47  Comprehensive  Training  Prograa 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

1 

3 

3 

Automated  Procedure 

10 

2 

20 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Environment  Compatibility 
Conceptual  Simplicity 

8 

3 

24 

Use 

10 

2 

20 

TOTAL 

127.0 

NEED  HEIGHT 

*  3.B 

CIS  SCORE 

482.6 

CONCEPT  III  PL  EHE  NT  ATI  ON  EVALUATION 

SHEET 

o  IBPLEBENTATION:  USBIT 

• 

• 

o  FOB  CONCEPT:  *11  Sapid  Prototyping 

• 

o  SATISFIES  NEED:  *21  Simulator  for 

Design 

EVALUATION  CRITERIA 

EVALUATION  X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

Diagnostics 

U 

2 

8 

Docuaentation 

5 

2 

10 

Interactive  Support 

5 

2 

10 

Autoaated  Procedure 

B 

2 

16 

Hatur ity 

3 

3 

9 

Vendor  Support 

7 

1 

7 

Availability 

10 

3 

30 

Hardware  Coapatibility 

6 

3 

16 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

5 

2 

10 

Software 

10 

2 

20 

Conceptual  Siaplicity 

Use 

5 

2 

10 

Training 

3 

3 

9 

Output 

DMA  Applicable 

8 

3 

2U 

Understandable 

8 

2 

16 

Systea  Resources 

Allocations  Required 

6 

3 

18 

TOTAL 

250.0 

NEED 

HEIGHT 

x  1.6 

CIB  SCOHE 

uoo.o 

CONCEPT  III  PRESENTATION  EVALUATION  SHEET 
O  IHPLEHENTATION:  CS4 
o  FOB  CONCEPT:  *11  Rapid  Prototyping 
o  SATISFIES  NEED:  *21  Siaulator  for  Design 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 
Support  Doeuaentation 
Diagnostics 

Doeuaentation 
Interactive  Support 
Autoaated  Procedure 
Haturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardvare 
Sof tvare 

Conceptual  Siaplicity 
Use 

Training 

Output 

DHA  Applicable 
Understandable 
Systaa  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 
CIB  SCORE 


1 

3 

3 

8 

2 

16 

5 

2 

10 

1 

2 

2 

5 

2 

10 

10 

3 

30 

2 

1 

2 

10 

3 

30 

7 

3 

21 

7 

1 

7 

e 

2 

16 

8 

2 

16 

7 

2 

11* 

8 

3 

24 

7 

3 

21 

0 

2 

0 

0 

3 

0 

222.0 
x  1.6 
355.2 


380 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  PAHS 
o  POB  CONCEPT:  *11  Hapid  Prototyping 
o  SATISFIES  NEED:  *21  Siaulator  for  Design 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 
Sof  tware 

Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
System  Resources 

Allocations  Required 


5  3  15 

3  2  6 


a 

8 

6 

2 

3 

10 

1 

5 


2 

2 

2 

3 

1 

3 

3 

1 


1 

1 

1 


30 

3 

5 


8  2  16 
8  2  16 


9  2  18 

9  3  27 


1  3  3 

1  2  2 


0 


0 


TOTAL 

NEED  HEIGHT 
Cl E  SCORE 


194.0 
x  1.6 
310.4 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  SHELL 
o  FOR  CONCEPT:  Ml  Rapid  Prototyping 
o  SATISFIES  NEED:  #22  Prograa  Design  Language 


.  EVALDATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 
Diagnostics 

5 

2 

10 

Docuaentation 

5 

2 

10 

Interactive  Support 

3 

2 

6 

Automated  Procedure 

10 

2 

20 

Maturity 

10 

3 

30 

Vendor  Support 

4 

1 

4 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

5 

2 

10 

Software 

Conceptual  Siaplicity 

1 

2 

2 

Use 

1 

2 

2 

Training 

Output 

2 

3 

6 

DHA  Applicable 

3 

3 

9 

Understandable 

Systea  Resources 

8 

2 

16 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIS  SCORE 

4 

3 

12 

232.0 
x  3.6 
835.2 

1 


■ 


1 


■ 
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CONCEPT  IH PLED ENT ATI ON  EVALUATION  SHEET 
o  IMPLEMENTATION:  USEIT 
o  FOB  CONCEPT:  *11  Sapid  Prototyping 
o  SATISFIES  NEED:  #22  Prograa  Design  Language 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

10 

3 

30 

Support  Docuaentation 
Diagnostics 

4 

2 

8 

Docuaentation 

5 

2 

10 

Interactive  Support 

5 

2 

10 

Automated  Procedure 

8 

2 

16 

Baturity 

3 

3 

9 

Vender  Support 

7 

1 

7 

Availability 

10 

3 

30 

Hardware  Coapatibility 

6 

3 

18 

Governaent  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

5 

2 

10 

Software 

Conceptual  Simplicity 

10 

2 

20 

Use 

5 

2 

10 

Traininq 

Output 

3 

3 

9 

DBA  Applicable 

8 

3 

24 

Understandable 

Systea  Resources 

8 

2 

16 

Allocations  '■equired 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

6 

3 

18 

250.0 
x  3.6 
900.0 
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CONCEPT  IflPLBHEHTATIOH  EVALUATION  SHEET 


o  IHPLEHENTATION:  CS4 
o  FOB  CONCEPT:  #11  Bapid  Prototyping 
o  SATISFIES  NEED:  #22  Prograa  Design  Language 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 
Support  Docunentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Autoaated  Procedure 
Haturity 
Vendor  Support 
Availability 
Hardware  Coapatibility 
Governaent  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Siaplicity 
Use 

Training 

Output 

DHA  Applicative 
Understandable 
Systea  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


1 

3 

3 

s 

2 

16 

5 

2 

10 

1 

2 

2 

5 

2 

10 

10 

3 

30 

2 

1 

2 

10 

3 

30 

7 

3 

21 

7 

1 

7 

8 

2 

16 

8 

2 

16 

7 

2 

14 

8 

3 

24 

7 

3 

21 

0 

2 

0 

0 

3 

0 

222.0 
x  3.6 

799.2 

4 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  OSEIT 
o  FOR  CONCEPT:  »11  Rapid  Prototyping 
o  SATISFIES  NEED:  *57  Software  Developaent  Tools 


EVALUATION  CRITERIA  .  EVALUATION  I  HEIGHT  =  SCOBE 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

Output 

DHA  Applicable 
Understandable 
System  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIS  SCOBE 


385 
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MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  of  STANDARDS  1%3-A 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION :  PAHS 
o  FOB  CONCEPT:  #11  Rapid  Prototyping 
o  SATISFIES  NEED:  #57  software  Development  Tools 


.  EVALUATION  CRITEBIA 

.  EVALUATION 

X  HEIGHT 

=  SCOBE 

Interactive  Capability 

5 

3 

15 

Support  Documentation 
Diagnostics 

3 

2 

6 

Docuaentation 

8 

2 

16 

Interactive  Support 

8 

2 

16 

Autoaated  Procedure 

6 

2 

12 

Haturity 

2 

3 

6 

Vendor  Support 

3 

1 

3 

A vailabilitY 

10 

3 

30 

Hardware  Coapatibility 

1 

3 

3 

Governaent  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

8 

2 

16 

Software 

8 

2 

16 

Conceptual  Simplicity 

Use 

9 

2 

18 

Training 

9 

3 

27 

Output 

DHA  Applicable 

1 

3 

3 

understandable 

1 

2 

2 

System  Resources 

Allocations  Required 

0 

3 

0 

TOTAL 

194.0 

NEED  HEIGHT 

x  4.2 

CIR  SCORE 

814.8 

CONCEPT  IHPLEHENTAT ION  EV ALU ATIOW  SHEET 


O  I HPLBHENTATION  :  CS4 
o  FOB  CONCEPT:  *11  Rapid  Prototyping 
o  SATISFIES  NEED:  *57  Software  Development  Tools 

EVALUATION  CRITEBIA  .  EVALUATION  X  HEIGHT  «  SCOBB 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Naturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

Output 

DBA  Applicable 
Understandable 
System  Resources 

Allocations  Bequired 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


1  3  3 


8 

2 

16 

5 

2 

10 

1 

2 

2 

5 

2 

10 

10 

3 

30 

2 

1 

2 

10 

3 

30 

7 

3 

21 

7 

1 

7 

8 

2 

16 

8 

2 

16 

7 

2 

14 

8 

3 

24 

7 

3 

21 

0 

2 

0 

0 

3 

0 

222.0 
x  4.2 

932.4 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  ASET 

o  FOB  CONCEPT:  #12  Automated  Training  Prograe 
o  SATISFIES  NEED:  *12  Improve  Banloading 


EVALUATION  CRITERIA 

• 

EVALOATION 

I  HEIGHT  = 

SCORE 

___ 

Interactive  Capability 

10 

3 

Support  Documentation 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Environment  Compatibility 

5 

3 

15 

Conceptual  Simplicity 

Dse 

8 

2 

16 

System  Resources 

Allocations  Required 

0 

3 

0 

TOTAL 

NEED  HEIGHT 
CIE  SCOHE 


131.0 
X  2.8 
366.8 


CONCEPT  IflPLEHENTATION  EVALUATION  SBEET 


o  IHPLEHENTATION:  ASET 

o  FOB  CONCEPT:  *12  Automated  Training  Program 
o  S1TISPIES  NEED:  *18  Paster  Intergation  of  New  Eepl's 


EVALUATION  CRITERIA 


Interactive  Capability 
Support  Documentation 
Maturity 
Availability 

Environment  Compatibility 
Conceptual  Simplicity 
Use 

System  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


EVALUATION  X  HEIGHT  =  SCORE 


10 

3 

30 

5 

2 

10 

10 

3 

30 

10 

3 

30 

5 

3 

15 

8 

2 

16 

0  3  0 

131.0 
x  2.B 
366.8 
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CONCEPT  IB PL EH ENT ATI ON  EVALUATION  SHEET 


o  IHPLEHENTATION:  ASET 

o  FOB  CONCEPT:  112  Automated  Training  Prograa 
o  SATISFIES  NEED:  *47  Comprehensive  Training  Prograa 

EVALUATION  CHITEBIA  .  EVALUATION  X  HEIGHT 


Interactive  Capability 

10 

3 

Support  Docuaentation 

5 

2 

Maturity 

10 

3 

Availability 

10 

3 

Environaent  Coapatibility 

5 

3 

Conceptual  Siaplicity 

Ose 

8 

2 

Systea  Resources 

Allocations  Required 

0 

3 

TOTAL 

NEED  HEIGHT 
CIE  SCOBE 


SCOBE 


30 

10 

30 

30 

15 

16 
0 

131.0 
x  3.8 
497.8 


CONCEPT  IMPLEMENTATION 

EVALUATION 

SHEET 

o  IMPLEMENTATION:  SOFTOOL  80 

o  FOB  CONCEPT:  *12  Automated 

Training  Prograa 

o  SATISFIES  NEED:  »47  Coaprehensi ve 

Training 

Prograa 

evaluation  CBITBBIA 

• 

EVALUATION  X  HEIGHT 

=  SCOBE 

— 

Interactive  Capability 

6 

3 

Support  Docuaentation 

5 

2 

10 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Environaent  coapatibility 
Conceptual  Siaplicity 

4 

3 

12 

Use 

6 

2 

12 

Systea  Besources 

Allocations  Bequired 

0 

3 

0 

TOTAL 

112.0 

SEED 

HEIGHT 

*  3.8 

CIB  SCOBE 

425.6 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  FAME 

o  FOB  CONCEPT:  113  Automated  Bequireaents  Generation 
o  SATISFIES  NEED:  *1  Foraal  Bequireaents  Specification 


EVALUATION  CBITEBI A 

.  EVALUATION  X 

HEIGHT  =  SCORE 

• 

Interactive  Capability 

10 

3 

30 

Support  Docuaentation 

5 

2 

10 

Diagnostics 

Docuaentation 

4 

2 

8 

Interactive  Support 

6 

2 

12 

Autoaated  Procedure 

7 

2 

14 

Maturity 

8 

3 

24 

Vendor  Support 

8 

1 

8 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environaent  Compatibility 

5 

3 

15 

Government  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

8 

2 

16 

Software 

10 

2 

20 

Conceptual  Siaplicity 

Use 

9 

2 

18 

Training 

6 

3 

16 

Output 

DMA  Applicable 

7 

3 

21 

Understandable 

8 

2 

16 

Systea  Resources 

Allocations  Required 

9 

3 

27 

TOTAL 

307.0 

NEED  HEIGHT 

l  2.6 

CIE  SCOBE 

798.2 

392 
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CONCEPT  IMPLEMENT AT ION  EVALOATION  SHEET 


.  o  IMPLEMENT AT I ON :  PSL/PSA 

.  o  FOB  CONCEPT:  *13  Automated 

.  o  SATISFIES  NEED:  *1  Foraal 

Requirements 

Requirements 

Generation 

Specification 

.  EVALOATION  CBITEBIA 

.  EVALOATION  X  WEIGHT 

=  SCOBE 

Interactive  Capability 

8 

3 

24 

Support  Documentation 

10 

2 

20 

Diagnostics 

Docuaentation 

8 

2 

16 

Interactive  Support 

0 

2 

0 

Automated  Procedure 

7 

2 

14 

Maturity 

9 

3 

27 

Vendor  Support 

7 

1 

7 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Environment  Compatibility 

5 

3 

15 

Government  Access 

5 

1 

5 

Flexibility  of  Ose 

Hardware 

6 

2 

12 

Software 

10 

2 

20 

conceptual  Simplicity 

Use 

9 

2 

18 

Training 

5 

3 

15 

Output 

DMA  Applicable 

7 

3 

21 

Understandable 

8 

2 

16 

System  aesources 

Allocations  Required 

7 

3 

21 

TOTAL 

311.0 

NEED  WEIGHT 

x  2.6 

CIS  SCOBE 

808.6 

393 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IMPLEMENTATION:  SBIMP 

o  FOB  CONCEPT:  *13  Automated  Requireaents  Generation 
o  SATISFIES  NEED:  *1  Foraal  Bequireaents  Specification 


EVALUATION  CBITEHIA 

.  EVALUATION 

I  WEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Docuaentation 
Diagnostics 

8 

2 

16 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Automated  Procedure 

7 

2 

14 

Maturity 

10 

3 

30 

Vendor  Support 

10 

1 

10 

Availability 

1 

3 

3 

Hardware  Coapatibility 

1 

3 

3 

Environaent  Coapatibility 

5 

3 

15 

Government  Access 
Flexibility  of  Use 

1 

1 

1 

Hardware 

1 

2 

2 

Sof tware 

Conceptual  Simplicity 

10 

2 

20 

Use 

9 

2 

18 

Training 

Output 

6 

3 

18 

DMA  Applicable 

7 

3 

21 

Understandable 

Systea  Besources 

8 

2 

16 

Allocations  Required 

TOTAL 

NEED  WEIGHT 

CIE  SCORE 

4 

3 

12 

229.0 
x  2.6 
595.4 

394 
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CONCEPT  IHPLEHEBTATION  EVALUATION  SHEET 


.  o  IHPLEBENTATION:  LABE 

.  o  FOB  CONCEPT:  *13  Automated 

.  o  SATISFIES  NEED:  il  Foraal 

Requireaents 

Requirements 

Generation 

Specification 

.  EVALUATION  CRITERIA 

• 

EVALUATION  X  HEIGHT 

=  SCORE 

Interactive  Capability 

8 

3 

2U 

Support  Documentation 

5 

2 

10 

Diagnostics 

Docuaentation 

U 

2 

8 

Interactive  Support 

6 

2 

12 

Autoaated  Procedure 

7 

2 

1« 

(laturity 

8 

3 

24 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coapatibility 

5 

3 

15 

Environaent  Coapatibility 

5 

3 

15 

Government  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

8 

2 

16 

Software 

10 

2 

20 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

6 

3 

18 

Output 

DHA  Applicable 

7 

3 

21 

Understandable 

8 

2 

16 

Systea  Resources 

Allocations  Required 

8 

3 

24 

TOTAL 

297.0 

NEED  HEIGHT 

x  2.6 

CIE  SCORE 

772.2 
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CONCEPT  IHPLS DENTATION  EVALOATION  SHEET 
o  IMPLEMENTATION:  BDP  1100 

o  FOB  CONCEPT:  113  Automated  Requirements  Generation 
o  SATISFIES  NEED:  *5  Requirements  Tracking 


EVALOATION  CRITEBIA 


EVALOATION  X  HEIGHT  =  SCORE 


Interactive  Capability  10  3 

Support  Docuaentation  0  2 

Diagnostics 

Docuaentation  0  2 

Interactive  Support  0  2 

Automated  Procedure  4  2 

Maturity  1  3 

Vendor  Support  10  1 

Availability  9  3 

Hardware  Compatibility  10  3 

Environaent  Compatibility  S  3 

Governaent  Access  5  1 

Flexibility  of  Ose 

Hardware  5  2 

Software  5  2 

Conceptual  Siaplicity 

Ose  0  2 

Training  0  3 

Output 

DMA  Applicable  3  3 

Onderstandable  0  2 

Systea  Resources 

Allocations  Required  5  3 


30 

0 

0 

0 

8 

3 

10 

27 

30 

15 

5 

10 

10 

0 

0 

9 

0 

15 


TOTAL 

REED  HEIGHT 
CIE  SCORE 


172.0 
x  2.2 
378.4 


396 


.  CONCEPT  IBPLEP1ENTATION  EVALOATION  SHEET  * 

•  • 

.  o  IBPLEHENTATION:  FAHE 

•  • 

•  o  FOR  CONCEPT:  #13  Automated  Requirements  Generation  • 

•  • 

.  o  SATISFIES  NEED:  #41  Organ,  Tools/Techniques  Interface 

•  • 

• 

EVALUATION  CRITERIA 

EVALUATION  X  WEIGHT  -  SCORE 

• 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

5 

2 

10 

Diagnostics 

Documentation 

4 

2 

8 

Interactive  Support 

6 

2 

12 

Automated  Procedure 

7 

2 

14 

Maturity 

B 

3 

24 

Vendor  Support 

8 

1 

8 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

5 

3 

15 

Government  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

B 

2 

16 

Software 

10 

2 

20 

Conceptual  Simplicity 

Use 

9 

2 

18 

Training 

6 

3 

18 

Output 

DMA  Applicable 

7 

3 

21 

understandable 

8 

2 

16 

System  Resources 

Allocations  Required 

9 

3 

27 

TOTAL 

307.0 

NEED 

WEIGHT 

X  3.4 

CIE  SCORE 

1043.8 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  HDP  1100 

o  FOB  CONCEPT:  #13  Automated  Requireaents  Generation 
o  SATISFIES  NEED:  *41  Organ.  Tools/Technigues  Interface 


EVALOATION  CRITERIA 


EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  10 

Support  Docuaentation  0 

Diagnostics 

Docuaentation  0 

Interactive  Support  0 

Autogated  Procedure  4 

Maturity  1 

Vendor  Support  10 

Availability  9 

Hardware  Coe patibility  10 

Environaent  Coapatibility  5 

Governaent  Access  5 

Flexibility  of  Use 

Hardware  5 

Software  5 

Conceptual  Siaplicity 

Use  0 

Training  0 

Output 

DMA  Applicable  3 

Understandable  0 

Systaa  Rasources 

Allocations  Required  5 


3  30 

2  0 


2 

2 

2 

3 

1 

3 

3 

3 

1 


0 

0 

s 

3 

10 

27 

30 

15 

5 


2  10 
2  10 


2  0 

3  0 


3  9 

2  0 

3  15 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


172.0 
X  3.4 
584.8 
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CONCEPT  IHPLEBBNTATION  EVALUATION  SHEET 
o  IHPLEBENTATION:  SBIHP 

o  FOB  CONCEPT:  *13  Automated  Bequiraaents  Generation 
o  SATISFIES  NEED:  *41  Organ.  Tools/Techniques  Interface 


.  EVALDATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

«  SCORE 

Interactive  Capability 

io 

3 

30 

Support  Docuaentation 
Diagnostics 

e 

2 

16 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoeated  Procedure 

7 

2 

14 

Haturity 

10 

3 

30 

Vendor  Support 

10 

1 

10 

Availability 

1 

3 

3 

Hardware  Coapatibility 

1 

3 

3 

Environaent  Coapatibility 

5 

3 

15 

Governaent  Access 

Flexibility  of  Use 

1 

1 

1 

Hardware 

1 

2 

2 

Software 

Conceptual  Siaplicity 

10 

2 

20 

Use 

9 

2 

18 

Training 

Output 

6 

3 

18 

DMA  Applicable 

7 

3 

21 

Understandable 

Systea  Resources 

B 

2 

16 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIB  SCORE 

4 

3 

12 

229.0 
x  3.4 
778.6 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IHPLBHBNTATION:  PSL/PSA 

o  FOB  CONCEPT:  *13  Autoaated  Requirements  Generation 
o  SATISFIES  NEED:  *41  Organ.  Tools/Techniques  Interface 


.  EVALUATION  CRITEHIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

8 

3 

24 

Support  Docuaentation 
Diagnostics 

10 

2 

20 

Docuaentation 

8 

2 

16 

Interactive  Support 

0 

2 

0 

Autoaated  procedure 

7 

2 

14 

Maturity 

9 

3 

27 

Vendor  Support 

7 

1 

7 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Bnvironaent  Coapatibility 

5 

3 

15 

Governaent  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

6 

2 

12 

Software 

Conceptual  Siaplicity 

10 

2 

20 

Use 

9 

2 

18 

Training 

Output 

5 

3 

15 

DHA  Applicable 

7 

3 

21 

Understandable 

Systea  Resources 

8 

2 

16 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

7 

3 

21 

311.0 
X  3.4 

1057.4 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  LANE 

o  PON  CONCEPT:  #13  Autoaated  Requireaents  Generation 
o  SATISFIES  NEED:  #41  Organ.  Tools/Techniques  Interface 


.  EVALUATION  CHITBBIA 

.  EVALUATION  X 

HEIGHT 

=  SCORE 

Interactive  Capability 

8 

3 

24 

Support  Docuaentation 
Diagnostics 

5 

2 

10 

Docuaentation 

U 

2 

8 

Interactive  Support 

6 

2 

12 

Autoaated  Procedure 

7 

2 

14 

Maturity 

8 

3 

24 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coapatibility 

5 

3 

15 

Environaant  Coapatibility 

5 

3 

15 

Governaent  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

8 

2 

16 

Software 

Conceptual  Siaplicity 

10 

2 

20 

Use 

8 

2 

16 

Training 

Output 

6 

3 

18 

DMA  Applicable 

7 

3 

21 

Understandable 

Systea  Resources 

8 

2 

16 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIR  SCORE 

8 

3 

24 

297.0 
x  3.4 
1009.8 

CONCEPT  IHPLEHEHTATION  EVALUATION  SHEET 


o  10 PL EB ENT AT ION :  PAflE 

o  FOR  CONCEPT:  *13  Automated  Requirements  Generation 
o  SATISFIES  NEED:  »57  Software  Development  Tools 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

5 

2 

10 

Diagnostics 

Documentation 

4 

2 

8 

Interactive  Support 

6 

2 

12 

Automated  Procedure 

7 

2 

14 

Haturity 

8 

3 

24 

Vendor  Support 

8 

1 

8 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

5 

3 

15 

Government  Access 

5 

1 

5 

Flexibility  of  use 

Hardware 

6 

2 

16 

Software 

10 

2 

20 

Conceptual  Simplicity 

Use 

9 

2 

18 

Training 

6 

3 

18 

Output 

DRA  Applicable 

7 

3 

21 

Understandable 

8 

2 

16 

System  Resources 

Allocations  Reguired 

9 

3 

27 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


307.0 
X  4.2 
1289.4 


Interactive  capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Maturity 
Vendor  Support 
Availability 
Hardware  Compatibility 
Environment  compatibility 
Government  Access 
Flexibility  of  Dse 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

Output 

DMA  Applicable 
Understandable 
System  Besources 

Allocations  Beguired 


TOTAL 

NEED  HEIGHT 
CIE  SCOBS 


172.0 
X  4.2 
722.4 


CONCEPT  If!  PLEA  ENT  ATI  ON  EVALUATION  SHEET 


o  IBPLEBENTATION :  PSL/PSA 

o  POB  CONCEPT:  *13  Automated  Requireaents  Generation 
o  SATISFIES  NEED:  »57  Software  Developeent  Tools 


.  EVALUATION  C8ITEBIA 

.  EVALUATION 

I  HEIGHT 

*  SCORE 

Interactive  Capability 

8 

3 

24 

Support  Docueentation 

10 

2 

20 

Diagnostics 

Docueentation 

8 

2 

16 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

7 

2 

14 

Baturity 

9 

3 

27 

Vendor  Support 

7 

1 

7 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Environaent  Coapatibility 

5 

3 

15 

Governaent  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

6 

2 

12 

software 

10 

2 

20 

Conceptual  Siaplicity 

Use 

9 

2 

18 

Training 

5 

3 

15 

Output 

DBA  Applicable 

7 

3 

21 

Understandable 

8 

2 

16 

Systea  Hesources 

Allocations  Required 

7 

3 

21 

TOTAL 

311.0 

NEED  WEIGHT 

x  4.2 

CIE  SCORE 

1306.2 

404 


¥ 


JWOPfil 


concept  implementation  evaluation  sheet 


o  IMPLEMENTATION:  SBIHP 

o  PON  CONCEPT:  *13  Automated  Requirements  Generation 
o  SATISFIES  NEED:  #57  Software  Development  Tools 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

*  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 
Diagnostics 

8 

2 

16 

Docueentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

7 

2 

14 

Maturity 

10 

3 

30 

Vendor  Support 

10 

1 

10 

Availability 

1 

3 

3 

Hardware  Compatibility 

1 

3 

3 

Environment  Compatibility 

5 

3 

15 

Government  Access 

Flexibility  of  Use 

1 

1 

1 

Hardware 

1 

2 

2 

Software 

Conceptual  Simplicity 

10 

2 

20 

Use 

9 

2 

18 

Training 

Output 

6 

3 

18 

DMA  Applicable 

7 

3 

21 

Understandable 

System  Resources 

8 

2 

16 

Allocations  Required 

TOTAL 

NEED  WEIGHT 

CIB  SCORE 

U 

3 

12 

229.0 
x  4.2 
961.8 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  LABE 

o  FOB  CONCEPT:  *13  Automated  Requireaents  Generation 
o  SATISFIES  NEED:  »57  Software  Developaent  Tools 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

8 

3 

24 

Support  Documentation 

5 

2 

10 

Diagnostics 

Documentation 

9 

2 

8 

Interactive  Support 

6 

2 

12 

Automated  Procedure 

7 

2 

14 

Maturity 

8 

3 

24 

Vendor  Support 

9 

1 

9 

Availability 

10 

3 

30 

Hardware  Coapatibilit y 

5 

3 

15 

Environaent  Coapatibility 

5 

3 

15 

Govecnaent  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

8 

2 

16 

Software 

10 

2 

20 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

6 

3 

18 

Output 

DMA  Applicable 

7 

3 

21 

Understandable 

8 

2 

16 

Systaa  Resources 

Allocations  Required 

8 

3 

24 

TOTAL 

NEED  HEIGHT 
CIS  SCORE 


297.0 
x  9.2 
1297.4 


CONCEPT  IMPLEMENTATION  EVALOATION  SHEET 
O  IMPLEMENTATION:  SODL 

o  PON  CONCEPT:  #14  Software  Design  Language 
o  SATISFIES  NEED:  #22  Prograa  Design  Language 

EVALOATION  CRITEBIA  .  EVALUATION  X  HEIGHT  *  SCORE 


Interactive  Capability  10 

Support  Documentation  10 

Diagnostics 

Documentation  4 

Interactive  Support  2 

Automated  Procedure  4 

Maturity  9 

Vendor  Support  9 

Availability  10 

Hardware  Compatibility  7 

Environment  compatibility  8 

Government  Access  10 

Flexibility  of  Ose 

Hardware  7 

Software  10 

Conceptual  Simplicity 

Ose  9 

Training  9 

Output 

DMA  Applicable  7 

Understandable  10 

System  Resources 

Allocations  Required  0 


3  30 

2  20 


2 

2 

2 

3 

1 

3 

3 

3 

1 


8 

4 

8 

27 

9 

30 

21 

24 

10 


2  14 

2  20 


2  18 

3  27 


3  21 

2  20 


3  0 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


311.0 
x  3.6 
1119.6 


CONCEPT  IBPLEHENT ATION  EVALUATION  SHEET 


O  IHPLBHENTATION:  PDL 

o  FOB  CONCEPT:  *14  Software  Design  Language 
o  SATISFIES  NEED:  *22  Prograa  Design  Language 


EVALUATION  CRITERIA  .  EVALDATION  X  HEIGHT  =  SCORE 


Interactive  Capability  2 

Support  Docuaentation  10 

Diagnostics 

Docuaentation  0 

Interactive  Support  1 

Autoaated  Procedure  2 

Haturity  10 

Vendor  Support  0 

Availability  10 

Hardware  Coapatibility  7 

Environaent  Coapatibility  8 

Governaent  Access  5 

Flexibility  of  Use 

Hardware  6 

Software  8 

Conceptual  Siaplicity 

Use  9 

Training  9 

Output 

DHA  Applicable  7 

Understandable  10 

Systea  Resources 

Allocations  Required  0 


3  6 

2  20 


2 

2 

2 

3 

1 

3 

3 

3 

1 


0 

2 

4 
30 

0 

30 

21 

24 

5 


2  12 
2  16 


2  18 

3  27 


3  21 

2  20 


3  0 


TOTAL 

NEED  HEIGHT 
CIR  SCORE 


256.0 
t  3.6 
921.6 


CONCEPT  IBPLBBENTATION  EVALUATION  SHEET 


"I 


o  IB PL EB ENT AT ION:  SDDL 

o  FOB  CONCEPT:  #14  Software  Design  Language 
o  SATISFIES  NEED:  #41  Organ.  Tools/Techniques  Interface 


EVALUATION  CRITERIA 


EVALUATION  X  HEIGHT  *  SCORE 


Interactive  Capability  10 

Support  Documentation  10 

Diagnostics 

Docunentation  4 

Interactive  Support  2 

Autoeated  Procedure  4 

Baturity  9 

Vendor  Support  9 

Availability  10 

Hardware  Coapatibility  7 

Environment  Coapatibility  8 

Government  Access  10 

Flexibility  of  Use 

Hardware  7 

software  10 

Conceptual  Simplicity 

Use  9 

Training  9 

Output 

DBA  Applicable  7 

Understandable  10 

System  Resources 

Allocations  Required  0 


3  30 

2  20 


2 

2 

2 

3 

1 

3 

3 

3 

1 


6 

4 

8 

27 

9 

30 

21 

24 

10 


2  14 

2  20 

2  18 

3  27 


3  21 

2  20 


3  0 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


311.0 
x  3.4 
1057.4 


409 
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CONCEPT  IB PLEA ENT ATI ON  EVALUATION  SHEET 


o  I HPL EH ENT AT I ON:  PDI 

o  FOB  CONCEPT:  *14  Software  Design  Language 
o  SATISFIES  NEED:  141  Organ.  Tools/Technigues  Interface 

EVALUATION  CBITERI A  .  EVALUATION  X  HEIGHT  -  SCORE 


Interactive  Capability  2 

Support  Documentation  10 

Diagnostics 

Documentation  0 

Interactive  Support  1 

Automated  Procedure  2 

Haturity  10 

Vendor  Support  0 

Availability  10 

Hardware  Compatibility  7 

Environment  Compatibility  0 

Government  Access  5 

Flexibility  of  Use 

Hardware  6 

Software  8 

Conceptual  Simplicity 

Use  9 

Training  9 

Output 

DBA  Applicable  7 

Understandable  10 

System  Resources 

Allocations  Required  0 


3  6 

2  20 


2 

2 

2 

3 

1 

3 

3 

3 

1 


0 

2 

4 
30 

0 

30 

21 

24 

5 


2  12 
2  16 


2  18 

3  27 


3  21 

2  20 


3  0 


TOTAL 

NEED  HEIGHT 
CIS  SCORE 


256.0 
x  3.4 
870.4 


410 
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CONCEPT  IHPLEBENTATION  EVALUATION  SHEET 
o  IHPLEBENTATION:  USER  INTERFACE  FOR  ON-LINE  ASSISTANCE 
o  FOB  CONCEPT:  *14  Software  Design  Language 
o  SATISFIES  NEED:  *54  Natural  Lang.  Oser/Sys.  Interface 

EVALOATION  CRITERIA  .  EVALUATION  l  HEIGHT  -  SCORE 


Interactive  Capability  To 

Support  Documentation  0 

Diagnostics 

Docuaentation  0 

Interactive  Support  10 

Autoaated  Procedure  10 

Maturity  1 

Vendor  Support  0 

Availability  1 

Hardware  Coapatibility  10 

Environaent  Coapatibility  1 

Government  Access  0 

Flexibility  of  Use 

Hardware  1 

Software  5 

Conceptual  Siaplicity 

Use  10 

Training  10 

Output 

DHA  Applicable  0 

Understandable  10 

Systea  Resources 

Allocations  Beguired  0 


3 

2 

2 

2 

2 

3 

1 

3 

3 

3 

1 

2 

2 

2 

3 

3 

2 

3 


30 

0 

0 

20 

20 

3 

0 

3 

30 

3 

0 

2 

10 

20 

30 

0 

20 

0 


TOTAL 

NEED  HEIGHT 
CIS  SCORE 


191.0 
X  1.4 
267.4 


411 


CONCEPT  I HPLE DENTITION  EVALUATION  SHEET 


O  ID PLED ENT AT 10 N :  SDDL 

o  FOB  CONCEPT:  *14  software  Design  Language 
o  SATISFIES  NEED:  #57  Software  Development  Tools 


EVALUATION  CBITEBIA  .  EVALUATION  X  HEIGHT  «  SCOBE 


Interactive  Capability 
Support  Documentation 
Diagnostics 

Documentation 
Interactive  Support 
Automated  Procedure 
Baturity 
Vendor  Support 
Availabili ty 
Hardware  Compatibility 
Environment  compatibility 
Government  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Simplicity 
Use 

Training 

Output 

DBA  Applicable 
Understandable 
System  Resources 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 


412 


* 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IHPLEHENT1TION:  PDL 

o  FOB  CONCEPT:  *14  Software  Design  Language 
o  SATISFIES  NEED:  *57  Software  Developaent  Tools 


EVALUATION  CRITERIA 


EVALDATION  X  HEIGHT  = 


SCORE 


Interactive  Capability  2 

Support  Docuaentation  10 

Diagnostics 

Docuaentation  0 

Interactive  Support  1 

Automated  Procedure  2 

Maturity  10 

Vendor  Support  0 

Availability  10 

Hardware  coapatibility  7 

Environment  Coapatibility  8 

Governaent  Access  5 

Flexibility  of  Use 

Hardware  6 

Software  8 

Conceptual  Simplicity 

Use  9 

Training  9 

Output 

DMA  Applicable  7 

Understandable  10 

Systea  Resources 

Allocations  Beguired  0 


3  6 

2  20 


2 

2 

2 

3 

1 

3 

3 

3 

1 


0 

2 

4 
30 

0 

30 

21 

24 

5 


2  12 
2  16 


2  18 

3  27 


3  21 

2  20 


3  0 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


256.0 
x  4.2 
1075.2 


CONCEPT  IHPLBHENTATION  EVALUATION  SHEET 
O  IHPLEHBNTATION:  A  OF 

o  FOR  CONCEPT:  *14  Software  Design  Language 
o  SATISFIES  NEED:  *5/  Software  Development  Tools 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability 

0 

3 

0 

Support  Docunentation 
Diagnostics 

0 

2 

0 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

5 

2 

10 

Maturity 

0 

3 

0 

Vendor  Support 

0 

1 

0 

Availability 

10 

3 

30 

Hardware  Coapatibility 

5 

3 

15 

Environaent  Compatibility 

5 

3 

15 

Government  Access 

Flexibility  of  Use 

5 

1 

5 

Hardware 

3 

2 

6 

Software 

Conceptual  Simplicity 

10 

2 

20 

Use 

4 

2 

8 

Training 

Output 

4 

3 

12 

DMA  Applicable 

5 

3 

15 

Understandable 

System  Resources 

0 

2 

0 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

C IE  SCORE 

9 

3 

27 

163.0 
x  4.2 
684.6 

414 


CONCEPT  I8PLEHENTATI0N  EVALUATION  SHEET 
o  IB PL EH ENT AT LON:  IPP 

o  FOB  CONCEPT:  #15  Structured  Programming  Facility 
o  SATISFIES  NEED:  *18  Faster  Intergation  of  New  Eapl's 

BVALOATION  CBITBRIA  .  EVALUATION  X  HEIGHT  «  SCORE 


Support  Documentation 
Diagnostics 

0 

2 

0 

Docueentation 

0 

2 

0 

Interactive  Support 

3 

2 

6 

Haturity 

i 

3 

3 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Government  Access 

Flexibility  of  Ose 

5 

1 

5 

Hardware 

5 

2 

10 

Software 

Conceptual  Simplicity 

5 

2 

10 

Ose 

6 

2 

12 

Training 

System  Resources 

0 

3 

0 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

6 

3 

18 

12W.0 
x  2-2 
272.8 

415 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  IPP 

o  FOB  CONCEPT:  IIS  Structured  Programming  Facility 
o  SATISFIES  NEED:  *30  Automated  Text  Management  System 


EVALOATION  CRITERIA  .  EVALUATION  I  HEIGHT  «  SCOBE 


Support  Documentation  0 

Diagnostics 

Documentation  0 

Interactive  Support  3 

Maturity  1 

Availability  10 

Hardware  Compatibility  10 

Government  Access  5 

Flexibility  of  Use 

Hardware  5 

Software  5 

Conceptual  Simplicity 

Use  6 

Training  0 

System  Resources 

Allocations  Beguired  6 


2  0 


2 

2 

3 

3 

3 

1 


0 

6 

3 

30 

30 

5 


2  10 
2  10 


2  12 

3  0 

3  18 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


124.0 
X  3.8 
471.2 


CONCEPT  IHPLEBENTATION  EVALUATION  SHEET 


O  Id PL EH ENT AT ION:  IPP 

o  FOB  CONCEPT:  115  Structured  Progressing  Facility 
o  SATISFIES  NEED:  #55  Hodern  Source  Data  Entry  Tech's 


.  EVALUATION  CRITERIA 

EVALUATION 

X  HEIGHT 

»  SCORE 

Support  Docuaentation 
Diagnostics 

0 

2 

0 

Docuaentation 

0 

2 

0 

Interactive  Support 

3 

2 

6 

Maturity 

1 

3 

3 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Government  Access 

Flexibility  of  use 

5 

1 

5 

Hardware 

5 

2 

10 

Software 

Conceptual  Siaplicity 

5 

2 

10 

Use 

6 

2 

12 

Training 

Systea  Resources 

0 

3 

0 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

6 

3 

18 

124.0 
x  4.6 
570.4 

CONCEPT  IHPLEHSNTATION  EVALUATION  SHEET 
o  IBPLEBENTATION:  IP? 

o  FOB  CONCEPT:  *15  Structured  Progressing  Facility 
o  SATISFIES  NEED:  *57  Software  Development  Tools 


EVALUATION  CRITEHIA 

.  EVALUATION 

I  HEIGHT 

*  SCORE 

Support  Docuaentation 

6 

2 

0 

Diagnostics 

Docuaentation 

0 

2 

0 

Interactive  Support 

3 

2 

6 

Raturity 

1 

3 

3 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Government  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

5 

2 

10 

Software 

5 

2 

10 

Conceptual  Siaplicity 

Use 

6 

2 

12 

Training 

0 

3 

0 

Systea  Resources 

Allocations  Required 

6 

3 

18 

TOTAL 

NEED  HEIGHT 
CIE  SCOBS 


124.0 
x  4.2 
520.8 


CONCEPT  IHPLEHENTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  OTS  4000  TEXT  PB0CESS08 
o  FOB  CONCEPT:  *16  Interactive  Text  Processing 
o  SBTtSFISS  NEED:  *11  Decreased  Paperwork 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  *  SCORE 


Support  Docueentation 
Diagnostics 

i~ 

2 

10 

Docueentation 

0 

2 

0 

Interactire  Support 

0 

2 

0 

Maturity 

1 

3 

3 

Availability 

10 

3 

30 

Hardware  Coepatibility 

10 

3 

30 

Environeent  Coepatibility 

6 

3 

18 

Government  Access 

Flexibility  of  Use 

5 

1 

S 

Hardware 

5 

2 

10 

Software 

Concaptual  Simplicity 

5 

2 

10 

Use 

6 

2 

12 

Training 

Output 

3 

3 

9 

DMA  Applicable 

Systee  Resources 

7 

3 

21 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIS  SCORE 

9 

3 

27 

185.0 
x  2.0 
370.0 

419 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  IS/1  INWOrd 
o  POE  CONCEPT:  #16  Interactive  Text  Processing 
o  SATISFIES  NEED:  til  Decreased  Paperwork 

EVALUATION  CHITEBIA  .  EVALUATION  X  HEIGHT  *  SCOEE 


Support  Docuaentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Maturity 
Availability 
Hardware  Coapatibility 
Environaent  Coapatibility 
Governaent  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Sieplicity 
Use 

Training 

Output 

DMA  Applicable 
Systea  Resources 

Allocations  Required 


8 

8 

8 

9 

10 

3 

U 

5 

3 

1 

8 

7 

9 


2  16 


2 

2 

3 

3 

3 

3 

1 


16 

16 

27 

30 

9 

12 

5 


2  6 
2  2 


2  16 

3  21 


3  12 

3 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


188.0 
X  2.0 
376.0 


420 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IMPLEMENTATION:  UTS  4000  TEXT  PHOCESSOB 
o  POM  CONCEPT:  *16  Interactive  Text  Processing 
o  SATISFIES  NSEC:  *34  Antosated  Text  Hanageaent  systea 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

*  SCORE 

Support  Docuaentation 

5 

2 

10 

Diagnostics 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Maturity 

1 

3 

3 

Availability 

10 

3 

30 

Hardvare  Coapatibility 

10 

3 

30 

Environaent  Coapatibility 

6 

3 

18 

Governaent  Access 

5 

1 

5 

Flexibility  of  Use 

Hardvare 

5 

2 

10 

Software 

5 

2 

10 

Conceptual  Siaplicity 

Use 

6 

2 

12 

Training 

3 

3 

9 

Output 

DMA  Applicable 

7 

3 

21 

Systea  Resources 

Allocations  Required 

9 

3 

27 

TOTaL 

185.0 

NEED  HEIGHT 

x  3.8 

CIS  SCORE 

703.0 

421 


Uv*  >?* 


ft 


...  i 
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Support  Docuaentation 
Diagnostics 

Docuaentation 
Interactive  Support 
Maturity 
availability 
Hardware  Coapatibility 
Environaent  Coapatibility 
Governaent  Access 
Flexibility  of  Use 
Hardware 
Software 

Conceptual  Siaplicity 
Use 

Training 

Output 

DHA  Applicable 
Systea  Resources 

Allocations  Required 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


188.0 
X  3.8 
714.4 


CONCEPT  INPLEHENTATION  EVALUATION  SHEET 


o  INPLEHENTATION:  The  Daily  Planlt 
o  FOB  CONCEPT:  *17  Autoaated  Data  Collaction 
o  SATISFIES  NEED:  *11  Decreased  Paperwork 


EVALUATION  CRITERIA 


EVALUATION  X  WEIGHT  *  SCORE 


Interactive  Capability 
Support  Docuaentation 
Maturity 
Availability 
Hardware  Coapatibility 
Environaent  coapatibility 
Governaent  Access 
Flexibility  of  Use 
Hardware 
Software 

conceptual  Siaplicity 
Use 

Training 

Output 

DHA  Applicable 
Understandable 
Systea  Resources 

Allocations  Required 


1  3 

8  2 

8  3 

10  3 

10  3 

7  3 

5  1 


3 

16 

24 

30 

30 

21 

5 


U  2  8 

10  2  20 

7  2  14 

7  3  21 


8  3  24 

7  2  14 


1  3  3 


TOTAL 

NEED  WEIGHT 
CIE  SCORE 


233.0 
x  2.0 
466.0 


CONCEPT  t NPLEHESTATIOI  EVALUATION  SHEET 


o  IHPLENBNTATION:  The  Daily  Planlt 
o  POE  CONCEPT:  *17  Automated  Data  Collection 
o  SATISFIES  NEED:  #40  Historical  Data  Base  Techniques 

EVALUATION  CRITERIA  -  EVALUATION  X  HEIGHT  *  SCORE 

■w 


Interactive  Capability  1 

Support  Docueentation  8 

Haturity  8 

Availability  10 

Hardware  Compatibility  10 

Environeent  Compatibility  7 

Government  Access  5 

Flexibility  of  Ose 

Hardware  4 

Software  10 

Conceptual  Simplicity 

Use  7 

Training  7 

Output 

DMA  Applicable  8 

Understandable  7 

System  Resources 

Allocations  Required  1 


3 

2 

3 

3 

3 

3 

1 


3 

16 

24 

30 

30 

21 

5 


2  8 
2  20 


2  14 

3  21 


3  24 

2  14 

3  3 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


233.0 
X  2.6 
605.8 


424 


< 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  The  Daily  Planlt 
o  FOB  CONCEPT:  *17  Automated  Data  Collection 

o  SATISFIES  NEED:  *46  deduce  Accounting  Data  Report  Anoaalies 


EVALUATION  CRITERIA 


EVALUATION  I  HEIGHT  =  SCORE 


Interactive  Capability  1 

Support  Documentation  8 

Maturity  8 

Availability  10 

Hardware  compatibility  10 

Environment  Compatibility  7 

Government  Access  5 

Flexibility  of  Use 

Hardware  4 

software  10 

Conceptual  Simplicity 

Use  7 

Training  7 

Output 

DMA  Applicable  8 

Understandable  7 

System  Resources 

Allocations  Reguired  1 


3 

2 

3 

3 

3 

3 

1 

2 

2 

2 

3 

3 

2 

3 


3 

16 

24 

30 

30 

21 

5 

8 

20 

14 

21 

24 

14 

3 


TOTAL 

NEED  HEIGHT 
CIB  SCORE 


233.0 
x  2.8 
652.4 


425 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IflPLBHENTATION:  PAVS 

o  FOB  CONCEPT j  #19  Software  Testing  Systea 
o  SATISFIES  N'<!0:  #57  Software  Developaent  Tools 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

3 

3 

9 

Support  Docuaentation 

8 

2 

16 

Diagnostics 

Docuaentation 

7 

2 

14 

Interactive  Support 

1 

2 

2 

Autoaated  Procedure 

10 

2 

20 

Maturity 

8 

3 

24 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Environaant  Coapatibility 

8 

3 

24 

Governaent  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

5 

2 

10 

Software 

8 

2 

16 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Output 

DMA  Applicable 

8 

3 

24 

Understandable 

7 

2 

14 

Systea  Resources 

Allocations  Required 

8 

3 

24 

TOTAL 

NEED  HEIGHT 
CIE  SCORE 


307.0 
*  4.2 

1289.4 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION :  SCHS 

o  FOR  CONCEPT :  #19  Software  Testing  System 
o  SATISFIES  NEED:  #57  Software  Development  Tools 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

1 

3 

3 

Support  Documentation 

9 

2 

18 

Diagnostics 

Docuaentation 

6 

2 

12 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

9 

2 

18 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Conpatibility 

10 

3 

30 

Environment  Compatibility 

6 

3 

18 

Government  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

5 

2 

10 

Software 

7 

2 

1  u 

Conceptual  simplicity 

Use 

7 

2 

14 

Training 

1 

3 

3 

Output 

DMA  Applicable 

6 

3 

18 

Understandable 

9 

2 

18 

System  Resources 

Allocations  Required 

8 

3 

24 

TOTAL 

270.0 

NEED  WEIGHT 

x  4.2 

CIE  SCORE 

1134.0 

427 
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CONCEPT  IHPLEHEHTATIOB  EVALUATION  SHEET 


o  IHPLEHENTATION:  FORTRAN  77  ANALYZES 
o  PON  CONCEPT:  *19  Software  Testing  Systee 
o  SATISFIES  NEED:  *57  Software  Development  Tools 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  »  SCOBE 


Interactive  Capability 

3 

3 

9 

Support  Docuaentation 
Diagnostics 

0 

2 

0 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

10 

2 

20 

Maturity 

1 

3 

3 

Availability 

5 

3 

15 

Hardware  compatibility 

5 

3 

15 

Environaent  Compatibility 

6 

3 

18 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

6 

2 

12 

Software 

Conceptual  Siaplicity 

8 

2 

16 

Use 

8 

2 

16 

Training 

Output 

8 

3 

24 

DMA  Applicable 

6 

3 

18 

Understandable 

Systaa  Resources 

7 

2 

14 

Allocations  Required 

TOTAL 

NEED  HEIGHT 

CIS  SCORE 

8 

3 

24 

214.0 
x  4.2 
898.8 

CONCEPT  IHPLEBENTATION  EVALUATION  SHEET 
o  IHPLEHERTATION:  SOFTOOL  80 
o  FOS  CONCEPT:  *19  Software  Testing  Systea 
o  SATISFIES  NEED:  #57  Software  Development  Tools 


EVALUATION  CRITERIA 

• 

EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

10 

3 

30 

Support  Documentation 

10 

2 

20 

Diagnostics 

Docuaentation 

e 

2 

16 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

8 

2 

16 

Maturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Coapatibility 

4 

3 

12 

Government  Access 

5 

1 

5 

Flexibility  of  Use 

Hardware 

7 

2 

14 

Software 

7 

2 

14 

Conceptual  Simplicity 

Use 

6 

2 

12 

Training 

8 

3 

24 

Out  put 

DBA  Applicable 

5 

3 

15 

Understandable 

7 

2 

14 

Systea  Resources 

Allocations  Required 

0 

3 

0 

TOTAL 

NEED  WEIGHT 
CIE  SCORE 


267.0 
*  4.2 

1121.4 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  PATS 


o  FOR  CONCEPT :  #19  Software  Testing  Systea 


.  o  SATISFIES  NEED:  #58  Production 

Prograa 

Optiaization 

• 

• 

« 

.  EVALUATION  CRITERIA 

EVALUATION  X  HEIGHT 

>  SCORE 

Interactive  Capability 

3 

3 

9 

Support  Docunentation 

8 

2 

16 

Diagnostics 

Docuaentation 

7 

2 

14 

Interactive  Support 

1 

2 

2 

Autoaated  Procedure 

10 

2 

20 

Maturity 

8 

3 

24 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Environaent  coapatibility 

8 

3 

24 

Governaent  Access 

10 

1 

10 

Flexibility  of  use 

Hardware 

5 

2 

10 

Software 

8 

2 

16 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Output 

DHA  Applicable 

8 

3 

24 

Understandable 

7 

2 

14 

Systea  Resources 

Allocations  Required 

8 

3 

24 

TOTAL 

307.0 

NEED  HEIGHT 

X  4.0 

CIE  SCORE 

1228.0 

CONCEPT  IHPLEBENTATION  B9ALDATI0N  SHEET 
o  IHPLEBENTATION:  FORTRAN  7?  ANALYZER 

o  FOR  CONCEPT:  *19  Software  Testing  Systea 
o  SATISFIES  NEED:  *58  Production  Prograa  Optiaization 

EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  «  SCORE 


Interactive  Capability  ~  3 

Support  Docuaentation  0 

Diagnostics 

Docuaentation  0 

Interactive  Support  0 

Autoaated  Procedure  10 

Maturity  1 

Availability  5 

Hardware  Coapatibility  S 

Environaent  Coapatibility  6 

Governaent  Access  10 

Flexibility  of  Use 

Hardware  6 

Software  8 

Conceptual  Sieplicity 

Use  8 

Training  8 

Output 

DHA  Applicable  6 

Understandable  7 

Systea  Resources 

Allocations  Required  8 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


214.0 
X  4.0 
856.0 


431 
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CONCEPT  IBPLEBENTATION  EVALUATION  SHEET 


o  IBPLEBENTATION:  SCBS 

o  P08  CONCEPT:  *19  Software  Testing  Systes 
o  SATISFIES  NEED:  *58  Production  Program  Optimization 


.  EVALUATION  CRITERIA 

.  EVALUATION  X  HEIGHT 

«  SCORE 

Interactive  Capability 

1 

3 

3 

Support  Documentation 
Diagnostics 

9 

2 

18 

Documentation 

6 

2 

12 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

9 

2 

18 

Haturity 

10 

3 

30 

Availability 

10 

3 

30 

Hardware  Coepatibility 

10 

3 

30 

Environaent  Compatibility 

6 

3 

18 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

5 

2 

10 

Software 

Conceptual  Simplicity 

7 

2 

19 

Use 

7 

2 

14 

Training 

Output 

1 

3 

3 

DBA  Applicable 

6 

3 

18 

Understandable 

System  Resources 

9 

2 

18 

Allocations  Reguired 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

8 

3 

24 

270.0 
x  4.0 

1080.0 

432 
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CONCEPT  IMPLEMENTATION  ETALOATION  SHEET 


o  IMPLEMENTATION:  SOFTOOL  80 
o  POE  CONCEPT:  #19  Software  Testing  Systea 
o  SATISFIES  NEED:  #58  Production  Prograa  Optimization 


EVALUATION  CRITERIA  .  EVALDATION  X  HEIGHT  =*  SCORE 


Interactive  Capability  10 

Support  Docuaentation  10 

Diagnostics 

Docuaentation  6 

Interactive  Support  0 

Autoaated  Procedure  8 

Maturity  10 

Availability  10 

Hardware  Coepatibility  5 

Environaent  Coepatibility  4 

Governaent  Access  5 

Flexibility  of  Ose 

Hardware  7 

Software  7 

Conceptual  Siaplicity 

Ose  6 

Training  8 

Output 

DHA  Applicable  5 

Understandable  7 

Systea  Resources 

Allocations  Required  0 


3  30 

2  20 


2 

2 

2 

3 

3 

3 

3 

1 


16 

0 

16 

30 

30 

15 

12 

5 


2  14 

2  14 


2  12 

3  24 


3  15 

2  14 


3  0 


TOTAL 

NEED  HEIGHT 
CIE  SCORE 


267.0 
X  4.0 
1068.0 


433 


i 
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CONCEPT  IMPLEMENTATION  ETALOATIOM  SHEET 


O  IMPLEMENTATION:  CATS 

o  FOB  CONCEPT:  *19  Software  Testing  Systee 


.  o  SATISFIES  NEED:  *58  Production 

Program 

Optimization 

• 

.  EVALUATION  CBITBRIA 

.  EVALUATION  X  HEIGHT 

*  SCORE 

• 

1 

Interactive  Capability 

7 

3 

21 

1 

( 

Support  Documentation 

8 

2 

16 

Diagnostics 

! 

Documentation 

0 

2 

0 

j 

Interactive  Support 

3 

2 

6 

i 

Automated  Procedure 

10 

2 

20 

Maturity 

1 

3 

3 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

|  Environment  Compatibility 

6 

3 

18 

i  Government  Access 

10 

1 

10 

|  Flexibility  of  Dse 

1  Hardware 

0 

2 

0 

• 

I  Software 

3 

2 

6 

|  Conceptual  Simplicity 

1  Use 

9 

2 

18 

1  Training 

5 

3 

15 

1  Output 

I  DMA  Applicable 

6 

3 

18 

1  Understandable 

0 

2 

0 

1  System  Resources 

1  Allocations  Required 

0 

3 

0 

1 

'll  TOTAL 

196.0 

1  NEED  WEIGHT 

x  4.0 

1 

II  CIE  SCORE 

784.0 

i 

•  3 

1 

s 

1 

1 

■i 

434 

J 
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CONCEPT  IHPLEHENTATZOR  EVALUATION  SHEET 
o  IHPLEBENTATION:  FAVS 

o  FOB  CONCEPT:  *20  Software  Standardization 
o  SATISFIES  NEED:  #2  QA  Procedures  S  Guidelines 


.  EVALUATION  CRITERIA 

.  EVALUATION 

Z  HEIGHT 

»  SCORE 

• 

• 

Interactive  Capability 

3 

3 

9 

Support  Documentation 

8 

2 

16 

Diagnostics 

Docunentation 

7 

2 

14 

Interactive  Support 

1 

2 

2 

Autoaated  Procedure 

10 

2 

20 

Maturity 

8 

3 

24 

Vendor  Support 

5 

1 

5 

Availability 

10 

3 

30 

Hardware  Compatibility 

10 

3 

30 

Environment  Compatibility 

8 

3 

24 

Government  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

5 

2 

10 

Software 

8 

2 

16 

Conceptual  Simplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Output 

DMA  Applicable 

8 

3 

24 

Understandable 

7 

2 

14 

TOTAL 

288.0 

NEED  HEIGHT 

x  3.6 

CIB  SCORE 

1036.8 

( 

* 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  FORTRAN  77  ANALYZES 
o  FOB  CONCEPT:  <20  Software  Standardization 
o  SATISFIES  NEED:  *2  Qi  Procedures  B  Guidelines 


.  EVALUATION  CRITERIA 

.  EVALUATION  X  HEIGHT 

=  SCORE 

Interactive  Capability 

3 

3 

9 

Support  Docuaentation 

0 

2 

0 

Diagnostics 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autonated  Procedure 

10 

2 

20 

Maturity 

1 

3 

3 

Vendor  Support 

10 

1 

10 

Availability 

5 

3 

15 

Hardware  coepatibility 

5 

3 

15 

Environaent  Coapatibility 

6 

3 

18 

Governaent  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

6 

2 

12 

Software 

S 

2 

16 

Conceptual  Siaplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Output 

DMA  Applicable 

6 

3 

18 

Understandable 

7 

2 

14 

TOTAL 

200.0 

NEED  HEIGHT 

x  3.6 

CIE  SCORE 

720.0 

CONCEPT  IMPLEMENTATION  EV ALO ATION  SHEET 


o  IMPLEMENTATION:  CAPS 

o  PON  CONCEPT:  120  Software  Standardization 
o  SATISFIES  NEED:  #2  QA  Procedures  C  Guidelines 


.  EVALUATION  CRITERIA 

.  EVALDATION 

X  HEIGHT 

*  SCORE 

Interactive  Capability 

7 

3 

21 

Support  Documentation 
Diagnostics 

6 

2 

16 

Documentation 

0 

2 

0 

Interactive  Support 

3 

2 

6 

Autoeated  Procedure 

10 

2 

20 

Maturity 

1 

3 

3 

Vendor  Support 

5 

1 

5 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

6 

3 

18 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

0 

2 

0 

Software 

Conceptual  Simplicity 

3 

2 

6 

Use 

9 

2 

18 

Training 

Output 

5 

3 

15 

DMA  Applicable 

6 

3 

18 

Understandable 

TOTAL 

NEED  HEIGHT 

CIE  SCORE 

0 

2 

0 

201.0 
x  3.6 
723.6 

CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  FA V S 

o  FOB  CONCEPT:  *20  Software  Standardization 
o  SATISFIES  NEED:  *57  software  Developaent  Tools 


.  EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

Interactive  Capability 

3 

3 

9 

Support  Documentation 

8 

2 

16 

Diagnostics 

Documentation 

7 

2 

14 

Interactive  Support 

1 

2 

2 

Automated  Procedure 

10 

2 

20 

Maturity 

B 

3 

24 

Vendor  Support 

5 

1 

5 

Availability 

10 

3 

30 

Hardware  Coapatibility 

10 

3 

30 

Environaent  Compatibility 

8 

3 

24 

Government  Access 

10 

1 

10 

Flexibility  of  Use 

Hardware 

5 

2 

10 

Software 

8 

2 

16 

Conceptual  Simplicity 

Use 

8 

2 

16 

Traininq 

8 

3 

24 

Output 

DMA  Applicable 

8 

3 

24 

Understandable 

7 

2 

14 

TOTAL 

288.0 

NEED  HEIGHT 

x  4.2 

CIB  SCORE 

1209.6 

438 
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CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
o  IMPLEMENTATION:  FORTNAN  77  ANALYZER 
o  FOR  CONCEPT:  *20  Software  Standardization 
o  SATISFIES  NEED:  #57  Software  Development  Tools 


EVALUATION  CRITERIA 

.  EVALUATION 

X  HEIGHT 

=  SCORE 

• 

Interactive  Capability 

3 

3 

9 

Support  Docuaentation 

0 

2 

0 

Diagnostics 

Docuaentation 

0 

2 

0 

Interactive  Support 

0 

2 

0 

Autoaated  Procedure 

10 

2 

20 

Maturity 

1 

3 

3 

Vendor  Support 

10 

1 

10 

Availability 

5 

3 

15 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

6 

3 

18 

Governaent  Access 

10 

4 

10 

Flexibility  of  Use 

Hardware 

6 

2 

12 

Software 

8 

2 

16 

Conceptual  Simplicity 

Use 

8 

2 

16 

Training 

8 

3 

24 

Output 

DMA  Applicable 

6 

3 

18 

Understandable 

7 

2 

19 

TOTAL 

200.0 

NEED  WEIGHT 

X  4.2 

C I E  SCORE 

840.0 

439 


CONCEPT  IBPLEBENTATIOB  EVALUATION  SHEET 


O  IBPLBBRNTATION:  CATS 

o  FOB  CONCEPT:  #20  Software  Standardization 
o  SATISFIES  NEED:  *57  Software  Development  Tools 

EVALUATION  CRITERIA  .  EVALUATION  X  WEIGHT  =  SCORE 


Interactive  Capability 

7 

3 

21 

Support  Documentation 
Diagnostics 

B 

2 

16 

Documentation 

0 

2 

0 

Interactive  Support 

3 

2 

6 

Automated  Procedure 

10 

2 

20 

flaturity 

1 

3 

3 

Vendor  Support 

5 

1 

5 

Availability 

10 

3 

30 

Hardware  Compatibility 

5 

3 

15 

Environment  Compatibility 

6 

3 

18 

Government  Access 

Flexibility  of  Use 

10 

1 

10 

Hardware 

0 

2 

0 

software 

Conceptual  Simplicity 

3 

2 

6 

Use 

9 

2 

18 

Training 

Output 

5 

3 

15 

DMA  Applicable 

6 

3 

18 

Understandable 

TOTAL 

NEED  HEIGHT 

CIS  SCORE 

0 

2 

0 

201.0 
X  4.2 
844.2 

440 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


o  IMPLEMENTATION:  Planlt  Bill-back 
o  FOR  CONCEPT:  *21  Chargeback  Systea 
o  SATISFIES  NEED:  #48  Chargeback  Systea 


EVALUATION  CRITERIA  .  EVALUATION  X  HEIGHT  =  SCORE 


Interactive  Capability  0 

Support  Documentation  0 

Automated  Procedure  7 

Maturity  1 

Availability  3 

Hardware  Compatibility  10 

Environment  Compatibility  5 

Government  Access  5 

Flexibility  of  Use 

Hardware  1 

Software  10 

Conceptual  Simplicity 

Use  0 

Training  0 

Output 

DHA  Applicable  6 

Understandable  6 

Systea  Resources 

Allocations  Reguired  3 


3 

2 

2 

3 

3 

3 

-3 

1 


0 

0 

14 
3 
9 

30 

15 
5 


2  2 
2  20 


2  0 

3  0 


3  18 

2  12 

3  9 


TOTAL 

NSEO  HEIGHT 
CIS  SCORE 


137.0 
x  3.4 
465.8 


441 

\ 


■T' 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 


O  IMPLEMENTATION:  IDA  .  j 

o  FOB  CONCEPT:  *22  Structured  Programming  .  1 


.  o  SATISFIES  NEED:  *59  Standardized  Phased 

Development 

.  EVALOATION  CRITERIA 

.  EVALUATION  X  HEIGHT 

*  SCORE 

Support  Docuaentation 

10 

2 

20 

Autoaated  Procedure 

1 

2 

2 

Maturity 

1 

3 

3 

Availability 

2 

3 

6 

Environaent  Coapatibility 

3 

3 

9 

Government  Access 

10 

1 

10 

Flexibility  of  Use 

Software 

1 

2 

2 

Conceptual  Siaplicity 

Use 

6 

2 

12 

Training 

2 

3 

6 

Output 

DMA  Applicable 

9 

3 

27 

Understandable 

9 

2 

18 

TOTAL 

115.0 

NEED  HEIGHT 

x  3.6 

CIE  SCORE 

414.0 

! 


CONCEPT  IHPLEHEHTATION  EVALUATION  SHEET 
o  IHPLEHENTATION:  FBDSIH 

o  FOB  CONCEPT:  *23  User  Assistance  Function 
o  SATISFIES  NEED:  #18  Faster  Intergation  of  New  Eapl's 


.  EVALUATION  CRITERIA 

EVALUATION 

X  HEIGHT 

*  SCORE 

Interactive  Capability 

1 

3 

3 

Automated  Procedure 

1 

2 

2 

Envirounent  Coapatibility 

10 

3 

30 

Output 

DHA  Applicable 

10 

3 

30 

TOTAL 

65.0 

NEED  WEIGHT 

*  2.2 

CIE  SCORE 

1«3.0 

CONCEPT  IHPLEBENTATION  EVALUATION  SHEET 
o  IBPLBHERT ATION :  PEDSIB 

o  FOB  CONCEPT:  *23  User  Assistance  Function 
o  SATISFIES  NEED:  *42  Dser  Assistance  Function 


.  EVALUATION  CRITERIA 

• 

EVALUATION 

X  HEIGHT 

«  SCORE 

• 

• 

Interactive  Capability 

1 

3 

3 

Autoaated  Procedure 

1 

2 

2 

Environaent  Compatibility 

10 

3 

30 

Output 

DBA  Applicable 

10 

3 

30 

TOTAL 

65.0 

NEED  HEIGHT 

x  3.6 

CIB  SCORE 

234.0 

444 


CONCEPT  IMPLEMENTATION  EVALUATION  SHEET 
O  IHPLEHENTATION:  PEDSIH 

o  FOB  CONCEPT:  *23  User  Assistance  Function 
o  SATISFIES  NEED:  *44  Error  Bate  Standards 


.  EVALUATION  CRITERIA 

.  EVALUATION  X  HEIGHT 

=  SCORE 

Interactive  Capability 

1  3 

3 

Autoaated  Procedure 

1  2 

2 

Environaent  Conpatibility 

10  3 

30 

Output 

DMA  Applicable 

10  3 

30 

TOTAL 

65.0 

NEED  HEIGHT 

x  2.6 

CIE  SCOKE 

169.0 

445 


